Development of Some Spatial-domain Preprocessing and Post-processing Algorithms for Better 2-D Up-scaling

Image super-resolution is an area of great interest in recent years and is extensively used in applications like video streaming, multimedia, internet technologies, consumer electronics, display and printing industries. Image super-resolution is a process of increasing the resolution of a given image without losing its integrity. Its most common application is to provide better visual effect after resizing a digital image for display or printing. One of the methods of improving the image resolution is through the employment of a 2-D interpolation. An up-scaled image should retain all the image details with very less degree of blurring meant for better visual quality. In literature, many efficient 2-D interpolation schemes are found that well preserve the image details in the up-scaled images; particularly at the regions with edges and fine details. Nevertheless, these existing interpolation schemes too give blurring effect in the up-scaled images due to the high frequency (HF) degradation during the up-sampling process. Hence, there is a scope to further improve their performance through the incorporation of various spatial domain pre-processing, post-processing and composite algorithms. Therefore, it is felt that there is sufficient scope to develop various efficient but simple pre-processing, post-processing and composite schemes to effectively restore the HF contents in the up-scaled images for various online and off-line applications. An efficient and widely used Lanczos-3 interpolation is taken for further performance improvement through the incorporation of various proposed algorithms. The various pre-processing algorithms developed in this thesis are summarized here. The term pre-processing refers to processing the low-resolution input image prior to image up-scaling. The various pre-processing algorithms proposed in this thesis are: Laplacian of Laplacian based global pre-processing (LLGP) scheme; Hybrid global pre-processing (HGP); Iterative Laplacian of Laplacian based global pre-processing (ILLGP); Unsharp masking based pre-processing (UMP); Iterative unsharp masking (IUM); Error based up-sampling(EU) scheme. The proposed algorithms: LLGP, HGP and ILLGP are three spatial domain preprocessing algorithms which are based on 4th, 6th and 8th order derivatives to alleviate nonuniform blurring in up-scaled images. These algorithms are used to obtain the high frequency (HF) extracts from an image by employing higher order derivatives and perform precise sharpening on a low resolution image to alleviate the blurring in its 2-D up-sampled counterpart. In case of unsharp masking based pre-processing (UMP) scheme, the blurred version of a low resolution image is used for HF extraction from the original version through image subtraction. The weighted version of the HF extracts are superimposed with the original image to produce a sharpened image prior to image up-scaling to counter blurring effectively. IUM makes use of many iterations to generate an unsharp mask which contains very high frequency (VHF) components. The VHF extract is the result of signal decomposition in terms of sub-bands using the concept of analysis filter bank. Since the degradation of VHF components is maximum, restoration of such components would produce much better restoration performance. EU is another pre-processing scheme in which the HF degradation due to image upscaling is extracted and is called prediction error. The prediction error contains the lost high frequency components. When this error is superimposed on the low resolution image prior to image up-sampling, blurring is considerably reduced in the up-scaled images. Various post-processing algorithms developed in this thesis are summarized in following. The term post-processing refers to processing the high resolution up-scaled image. The various post-processing algorithms proposed in this thesis are: Local adaptive Laplacian (LAL); Fuzzy weighted Laplacian (FWL); Legendre functional link artificial neural network(LFLANN). LAL is a non-fuzzy, local based scheme. The local regions of an up-scaled image with high variance are sharpened more than the region with moderate or low variance by employing a local adaptive Laplacian kernel. The weights of the LAL kernel are varied as per the normalized local variance so as to provide more degree of HF enhancement to high variance regions than the low variance counterpart to effectively counter the non-uniform blurring. Furthermore, FWL post-processing scheme with a higher degree of non-linearity is proposed to further improve the performance of LAL. FWL, being a fuzzy based mapping scheme, is highly nonlinear to resolve the blurring problem more effectively than LAL which employs a linear mapping. Another LFLANN based post-processing scheme is proposed here to minimize the cost function so as to reduce the blurring in a 2-D up-scaled image. Legendre polynomials are used for functional expansion of the input pattern-vector and provide high degree of nonlinearity. Therefore, the requirement of multiple layers can be replaced by single layer LFLANN architecture so as to reduce the cost function effectively for better restoration performance. With single layer architecture, it has reduced the computational complexity and hence is suitable for various real-time applications. There is a scope of further improvement of the stand-alone pre-processing and postprocessing schemes by combining them through composite schemes. Here, two spatial domain composite schemes, CS-I and CS-II are proposed to tackle non-uniform blurring in an up-scaled image. CS-I is developed by combining global iterative Laplacian (GIL) preprocessing scheme with LAL post-processing scheme. Another highly nonlinear composite scheme, CS-II is proposed which combines ILLGP scheme with a fuzzy weighted Laplacian post-processing scheme for more improved performance than the stand-alone schemes. Finally, it is observed that the proposed algorithms: ILLGP, IUM, FWL, LFLANN and CS-II are better algorithms in their respective categories for effectively reducing blurring in the up-scaled images

3d Face Reconstruction And Emotion Analytics With Part-Based Morphable Models

3D face reconstruction and facial expression analytics using 3D facial data are new and hot research topics in computer graphics and computer vision. In this proposal, we first review the background knowledge for emotion analytics using 3D morphable face model, including geometry feature-based methods, statistic model-based methods and more advanced deep learning-bade methods. Then, we introduce a novel 3D face modeling and reconstruction solution that robustly and accurately acquires 3D face models from a couple of images captured by a single smartphone camera. Two selfie photos of a subject taken from the front and side are used to guide our Non-Negative Matrix Factorization (NMF) induced part-based face model to iteratively reconstruct an initial 3D face of the subject. Then, an iterative detail updating method is applied to the initial generated 3D face to reconstruct facial details through optimizing lighting parameters and local depths. Our iterative 3D face reconstruction method permits fully automatic registration of a part-based face representation to the acquired face data and the detailed 2D/3D features to build a high-quality 3D face model. The NMF part-based face representation learned from a 3D face database facilitates effective global and adaptive local detail data fitting alternatively. Our system is flexible and it allows users to conduct the capture in any uncontrolled environment. We demonstrate the capability of our method by allowing users to capture and reconstruct their 3D faces by themselves. Based on the 3D face model reconstruction, we can analyze the facial expression and the related emotion in 3D space. We present a novel approach to analyze the facial expressions from images and a quantitative information visualization scheme for exploring this type of visual data. From the reconstructed result using NMF part-based morphable 3D face model, basis parameters and a displacement map are extracted as features for facial emotion analysis and visualization. Based upon the features, two Support Vector Regressions (SVRs) are trained to determine the fuzzy Valence-Arousal (VA) values to quantify the emotions. The continuously changing emotion status can be intuitively analyzed by visualizing the VA values in VA-space. Our emotion analysis and visualization system, based on 3D NMF morphable face model, detects expressions robustly from various head poses, face sizes and lighting conditions, and is fully automatic to compute the VA values from images or a sequence of video with various facial expressions. To evaluate our novel method, we test our system on publicly available databases and evaluate the emotion analysis and visualization results. We also apply our method to quantifying emotion changes during motivational interviews. These experiments and applications demonstrate effectiveness and accuracy of our method. In order to improve the expression recognition accuracy, we present a facial expression recognition approach with 3D Mesh Convolutional Neural Network (3DMCNN) and a visual analytics guided 3DMCNN design and optimization scheme. The geometric properties of the surface is computed using the 3D face model of a subject with facial expressions. Instead of using regular Convolutional Neural Network (CNN) to learn intensities of the facial images, we convolve the geometric properties on the surface of the 3D model using 3DMCNN. We design a geodesic distance-based convolution method to overcome the difficulties raised from the irregular sampling of the face surface mesh. We further present an interactive visual analytics for the purpose of designing and modifying the networks to analyze the learned features and cluster similar nodes in 3DMCNN. By removing low activity nodes in the network, the performance of the network is greatly improved. We compare our method with the regular CNN-based method by interactively visualizing each layer of the networks and analyze the effectiveness of our method by studying representative cases. Testing on public datasets, our method achieves a higher recognition accuracy than traditional image-based CNN and other 3D CNNs. The presented framework, including 3DMCNN and interactive visual analytics of the CNN, can be extended to other applications

Novel Signal Reconstruction Techniques in Cyclotron Radiation Emission Spectroscopy for Neutrino Mass Measurement

The Project 8 experiment is developing Cyclotron Radiation Emission Spectroscopy (CRES) on the beta-decay spectrum of tritium for the measurement of the absolute neutrino mass scale. CRES is a frequency-based technique which aims to probe the endpoint of the tritium energy spectrum with a final target sensitivity of 0.04 eV, pushing the limits beyond the inverted mass hierarchy. A phased-approach experiment, both Phase I and Phase II efforts use a combination of 83mKr and molecular tritium T_2 as source gases. The technique relies on an accurate, precise, and well-understood reconstructed beta-spectrum whose endpoint and spectral shape near the endpoint may be constrained by a kinematical model which uses the neutrino mass m_beta as a free parameter. Since the decays in the last eV of the tritium spectrum encompass O(10^(-13)) of all decays and the precise variation of the spectrum, distorted by the presence of a massive neutrino, is fundamental to the measurement, reconstruction techniques which yield accurate measurements of the frequency (and therefore energy) of the signal and correctly classify signal from background are necessary. In this work, we discuss the open-problem of the absolute neutrino mass scale, the fundamentals of measurements tailored to resolve this, the underpinning and details of the CRES technology, and the measurement of the first-ever CRES tritium $\beta$-spectrum. Finally, we focus on novel reconstruction techniques at both the signal and event levels using machine learning algorithms that allow us to adapt our technique to the complex dynamics of the electron inside our detector. We will show that such methods can separate true events from backgrounds at \u3e 94% accuracy and are able to improve the efficiency of reconstruction when compared to traditional reconstruction methods by \u3e 23%

Investigation of Non-coherent Discrete Target Range Estimation Techniques for High-precision Location

Ranging is an essential and crucial task for radar systems. How to solve the range-detection problem effectively and precisely is massively important. Meanwhile, unambiguity and high resolution are the points of interest as well. Coherent and non-coherent techniques can be applied to achieve range estimation, and both of them have advantages and disadvantages. Coherent estimates offer higher precision but are more vulnerable to noise and clutter and phase wrap errors, particularly in a complex or harsh environment, while the non-coherent approaches are simpler but provide lower precision. With the purpose of mitigating inaccuracy and perturbation in range estimation, miscellaneous techniques are employed to achieve optimally precise detection. Numerous elegant processing solutions stemming from non-coherent estimate are now introduced into the coherent realm, and vice versa. This thesis describes two non-coherent ranging estimate techniques with novel algorithms to mitigate the instinct deficit of non-coherent ranging approaches. One technique is based on peak detection and realised by Kth-order Polynomial Interpolation, while another is based on Z-transform and realised by Most-likelihood Chirp Z-transform. A two-stage approach for the fine ranging estimate is applied to the Discrete Fourier transform domain of both algorithms. An N-point Discrete Fourier transform is implemented to attain a coarse estimation; an accurate process around the point of interest determined in the first stage is conducted. For KPI technique, it interpolates around the peak of Discrete Fourier transform profiles of the chirp signal to achieve accurate interpolation and optimum precision. For Most-likelihood Chirp Z-transform technique, the Chirp Z-transform accurately implements the periodogram where only a narrow band spectrum is processed. Furthermore, the concept of most-likelihood estimator is introduced to combine with Chirp Z-transform to acquire better ranging performance. Cramer-Rao lower bound is presented to evaluate the performance of these two techniques from the perspective of statistical signal processing. Mathematical derivation, simulation modelling, theoretical analysis and experimental validation are conducted to assess technique performance. Further research will be pushed forward to algorithm optimisation and system development of a location system using non-coherent techniques and make a comparison to a coherent approach

Deep Reinforcement Learning for the Design of Structural Topologies

Advances in machine learning algorithms and increased computational efficiencies have given engineers new capabilities and tools for engineering design. The presented work investigates using deep reinforcement learning (DRL), a subset of deep machine learning that teaches an agent to complete a task through accumulating experiences in an interactive environment, to design 2D structural topologies. Three unique structural topology design problems are investigated to validate DRL as a practical design automation tool to produce high-performing designs in structural topology domains. The first design problem attempts to find a gradient-free alternative to solving the compliance minimization topology optimization problem. In the proposed DRL environment, a DRL agent can sequentially remove elements from a starting solid material domain to form a topology that minimizes compliance. After each action, the agent receives feedback on its performance by evaluating how well the current topology satisfies the design objectives. The agent learned a generalized design strategy that produced topology designs with similar or better compliance minimization performance than traditional gradient-based topology optimization methods given various boundary conditions. The second design problem reformulates mechanical metamaterial unit cell design as a DRL task. The local unit cells of mechanical metamaterials are built by sequentially adding material elements according to a cubic Bezier curve methodology. The unit cells are built such that, when tessellated, they exhibit a targeted nonlinear deformation response under uniaxial compressive or tensile loading. Using a variational autoencoder for domain dimension reduction and a surrogate model for rapid deformation response prediction, the DRL environment was built to allow the agent to rapidly build mechanical metamaterials that exhibit a diverse array of deformation responses with variable degrees of nonlinearity. Finally, the third design problem expands on the second to train a DRL agent to design mechanical metamaterials with tailorable deformation and energy manipulation characteristics. The agent’s design performance was validated by creating metamaterials with a thermoplastic polyurethane (TPU) constitutive material that increased or decreased hysteresis while exhibiting the compressive deformation response of expanded thermoplastic polyurethane (E-TPU). These optimized designs were additively manufactured and underwent experimental cyclic compressive testing. The results showed the E-TPU and metamaterial with E-TPU target properties were well aligned, underscoring the feasibility of designing mechanical metamaterials with customizable deformation and energy manipulation responses. Finally, the agent\u27s generalized design capabilities were tested by designing multiple metamaterials with diverse desired loading deformation responses and specific hysteresis objectives. The combined success of these three design problems is critical in proving that a DRL agent can serve as a co-designer working with a human designer to achieve high-performing solutions in the domain of 2D structural topologies and is worthy of incorporation into a wide array of engineering design domains

Spatial Fluctuations in the Diffuse Cosmic X-Ray Background

The bright, essentially isotropic, X-ray sky flux above 2 keV yields information on the universe at large distances. However, a definitive understanding of the origin of the flux is lacking. Some fraction of the total flux is contributed by active galactic nuclei and clusters of galaxies, but less than one percent of the total is contributed by the or approximately 3 keV band resolved sources, which is the band where the sky flux is directly observed. Parametric models of AGN (quasar) luminosity function evolution are examined. Most constraints are by the total sky flux. The acceptability of particular models hinges on assumptions currently not directly testable. The comparison with the Einstein Observatory 1 to keV low flux source counts is hampered by spectral uncertainties. A tentative measurement of a large scale dipole anisotropy is consistent with the velocity and direction derived from the dipole in the microwave background. The impact of the X-ray anisotropy limits for other scales on studies of large-scale structure in the universe is sketched. Models of the origins of the X-ray sky flux are reviewed, and future observational programs outlined

Signal and Information Processing Methods for Embedded Robotic Tactile Sensing Systems

The human skin has several sensors with different properties and responses that are able to detect stimuli resulting from mechanical stimulations. Pressure sensors are the most important type of receptors for the exploration and manipulation of objects. In the last decades, smart tactile sensing based on different sensing techniques have been developed as their application in robotics and prosthetics is considered of huge interest, mainly driven by the prospect of autonomous and intelligent robots that can interact with the environment. However, regarding object properties estimation on robots, hardness detection is still a major limitation due to the lack of techniques to estimate it. Furthermore, finding processing methods that can interpret the measured information from multiple sensors and extract relevant information is a Challenging task. Moreover, embedding processing methods and machine learning algorithms in robotic applications to extract meaningful information such as object properties from tactile data is an ongoing challenge, which is controlled by the device constraints (power constraint, memory constraints, etc.), the computational complexity of the processing and machine learning algorithms, the application requirements (real-time operations, high prediction performance). In this dissertation, we focus on the design and implementation of pre-processing methods and machine learning algorithms to handle the aforementioned challenges for a tactile sensing system in robotic application. First, we propose a tactile sensing system for robotic application. Then we present efficient preprocessing and feature extraction methods for our tactile sensors. Then we propose a learning strategy to reduce the computational cost of our processing unit in object classification using sensorized Baxter robot. Finally, we present a real-time robotic tactile sensing system for hardness classification on a resource-constrained devices. The first study represents a further assessment of the sensing system that is based on the PVDF sensors and the interface electronics developed in our lab. In particular, first, it presents the development of a skin patch (multilayer structure) that allows us to use the sensors in several applications such as robotic hand/grippers. Second, it shows the characterization of the developed skin patch. Third, it validates the sensing system. Moreover, we designed a filter to remove noise and detect touch. The experimental assessment demonstrated that the developed skin patch and the interface electronics indeed can detect different touch patterns and stimulus waveforms. Moreover, the results of the experiments defined the frequency range of interest and the response of the system to realistic interactions with the sensing system to grasp and release events. In the next study, we presented an easy integration of our tactile sensing system into Baxter gripper. Computationally efficient pre-processing techniques were designed to filter the signal and extract relevant information from multiple sensor signals, in addition to feature extraction methods. These processing methods aim in turn to reduce also the computational complexity of machine learning algorithms utilized for object classification. The proposed system and processing strategy were evaluated on object classification application by integrating our system into the gripper and we collected data by grasping multiple objects. We further proposed a learning strategy to accomplish a trade-off between the generalization accuracy and the computational cost of the whole processing unit. The proposed pre-processing and feature extraction techniques together with the learning strategy have led to models with extremely low complexity and very high generalization accuracy. Moreover, the support vector machine achieved the best trade-off between accuracy and computational cost on tactile data from our sensors. Finally, we presented the development and implementation on the edge of a real–time tactile sensing system for hardness classification on Baxter robot based on machine and deep learning algorithms. We developed and implemented in plain C a set of functions that provide the fundamental layer functionalities of the Machine learning and Deep Learning models (ML and DL), along with the pre–processing methods to extract the features and normalize the data. The models can be deployed to any device that supports C code since it does not rely on any of the existing libraries. Shallow ML/DL algorithms for the deployment on resource–constrained devices are designed. To evaluate our work, we collected data by grasping objects of different hardness and shape. Two classification problems were addressed: 5 levels of hardness classified on the same objects’ shape, and 5 levels of hardness classified on two different objects’ shape. Furthermore, optimization techniques were employed. The models and pre–processing were implemented on a resource constrained device, where we assessed the performance of the system in terms of accuracy, memory footprint, time latency, and energy consumption. We achieved for both classification problems a real-time inference (< 0.08 ms), low power consumption (i.e., 3.35 μJ), extremely small models (i.e., 1576 Byte), and high accuracy (above 98%)

Advanced Sensing, Fault Diagnostics, and Structural Health Management

Advanced sensing, fault diagnosis, and structural health management are important parts of the maintenance strategy of modern industries. With the advancement of science and technology, modern structural and mechanical systems are becoming more and more complex. Due to the continuous nature of operation and utilization, modern systems are heavily susceptible to faults. Hence, the operational reliability and safety of the systems can be greatly enhanced by using the multifaced strategy of designing novel sensing technologies and advanced intelligent algorithms and constructing modern data acquisition systems and structural health monitoring techniques. As a result, this research domain has been receiving a significant amount of attention from researchers in recent years. Furthermore, the research findings have been successfully applied in a wide range of fields such as aerospace, manufacturing, transportation and processes