Robust and Efficient Data Clustering with Signal Processing on Graphs

Data is pervasive in today's world and has actually been for quite some time. With the increasing volume of data to process, there is a need for faster and at least as accurate techniques than what we already have. In particular, the last decade recorded the effervescence of social networks and ubiquitous sensing (through smartphones and the Internet of Things). These phenomena, including also the progresses in bioinformatics and traffic monitoring, pushed forward the research on graph analysis and called for more efficient techniques. Clustering is an important field of machine learning because it belongs to the unsupervised techniques (i.e., one does not need to possess a ground truth about the data to start learning). With it, one can extract meaningful patterns from large data sources without requiring an expert to annotate a portion of the data, which can be very costly. However, the techniques of clustering designed so far all tend to be computationally demanding and have trouble scaling with the size of today's problems. The emergence of Graph Signal Processing, attempting to apply traditional signal processing techniques on graphs instead of time, provided additional tools for efficient graph analysis. By considering the clustering assignment as a signal lying on the nodes of the graph, one may now apply the tools of GSP to the improvement of graph clustering and more generally data clustering at large. In this thesis, we present several techniques using some of the latest developments of GSP in order to improve the scalability of clustering, while aiming for an accuracy resembling that of Spectral Clustering, a famous graph clustering technique that possess a solid mathematical intuition. On the one hand, we explore the benefits of random signal filtering on a practical and theoretical aspect for the determination of the eigenvectors of the graph Laplacian. In practice, this attempt requires the design of polynomial approximations of the step function for which we provided an accelerated heuristic. We used this series of work in order to reduce the complexity of dynamic graphs clustering, the problem of defining a partition to a graph which is evolving in time at each snapshot. We also used them to propose a fast method for the determination of the subspace generated by the first eigenvectors of any symmetrical matrix. This element is useful for clustering as it serves in Spectral Clustering but it goes beyond that since it also serves in graph visualization (with Laplacian Eigenmaps) and data mining (with Principal Components Projection). On the other hand, we were inspired by the latest works on graph filter localization in order to propose an extremely fast clustering technique. We tried to perform clustering by only using graph filtering and combining the results in order to obtain a partition of the nodes. These different contributions are completed by experiments using both synthetic datasets and real-world problems. Since we think that research should be shared in order to progress, all the experiments made in this thesis are publicly available on my personal Github account

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

Nonnegative matrix factorization for coherent set identification by direct low rank maximum likelihood estimation

We analyze connections between two low rank modeling approaches from the last decade for treating dynamical data. The first one is the coherence problem (or coherent set approach), where groups of states are sought that evolve under the action of a stochastic matrix in a way maximally distinguishable from other groups. The second one is a low rank factorization approach for stochastic matrices, called Direct Bayesian Model Reduction (DBMR), which estimates the low rank factors directly from observed data. We show that DBMR results in a low rank model that is a projection of the full model, and exploit this insight to infer bounds on a quantitative measure of coherence within the reduced model. Both approaches can be formulated as optimization problems, and we also prove a bound between their respective objectives. On a broader scope, this work relates the two classical loss functions of nonnegative matrix factorization, namely the Frobenius norm and the generalized Kullback–Leibler divergence, and suggests new links between likelihood- based and projection-based estimation of probabilistic models

Sparse and Redundant Representations for Inverse Problems and Recognition

Sparse and redundant representation of data enables the description of signals as linear combinations of a few atoms from a dictionary. In this dissertation, we study applications of sparse and redundant representations in inverse problems and object recognition. Furthermore, we propose two novel imaging modalities based on the recently introduced theory of Compressed Sensing (CS). This dissertation consists of four major parts. In the first part of the dissertation, we study a new type of deconvolution algorithm that is based on estimating the image from a shearlet decomposition. Shearlets provide a multi-directional and multi-scale decomposition that has been mathematically shown to represent distributed discontinuities such as edges better than traditional wavelets. We develop a deconvolution algorithm that allows for the approximation inversion operator to be controlled on a multi-scale and multi-directional basis. Furthermore, we develop a method for the automatic determination of the threshold values for the noise shrinkage for each scale and direction without explicit knowledge of the noise variance using a generalized cross validation method. In the second part of the dissertation, we study a reconstruction method that recovers highly undersampled images assumed to have a sparse representation in a gradient domain by using partial measurement samples that are collected in the Fourier domain. Our method makes use of a robust generalized Poisson solver that greatly aids in achieving a significantly improved performance over similar proposed methods. We will demonstrate by experiments that this new technique is more flexible to work with either random or restricted sampling scenarios better than its competitors. In the third part of the dissertation, we introduce a novel Synthetic Aperture Radar (SAR) imaging modality which can provide a high resolution map of the spatial distribution of targets and terrain using a significantly reduced number of needed transmitted and/or received electromagnetic waveforms. We demonstrate that this new imaging scheme, requires no new hardware components and allows the aperture to be compressed. Also, it presents many new applications and advantages which include strong resistance to countermesasures and interception, imaging much wider swaths and reduced on-board storage requirements. The last part of the dissertation deals with object recognition based on learning dictionaries for simultaneous sparse signal approximations and feature extraction. A dictionary is learned for each object class based on given training examples which minimize the representation error with a sparseness constraint. A novel test image is then projected onto the span of the atoms in each learned dictionary. The residual vectors along with the coefficients are then used for recognition. Applications to illumination robust face recognition and automatic target recognition are presented

Control of chaos in nonlinear circuits and systems

Nonlinear circuits and systems, such as electronic circuits (Chapter 5), power converters (Chapter 6), human brains (Chapter 7), phase lock loops (Chapter 8), sigma delta modulators (Chapter 9), etc, are found almost everywhere. Understanding nonlinear behaviours as well as control of these circuits and systems are important for real practical engineering applications. Control theories for linear circuits and systems are well developed and almost complete. However, different nonlinear circuits and systems could exhibit very different behaviours. Hence, it is difficult to unify a general control theory for general nonlinear circuits and systems. Up to now, control theories for nonlinear circuits and systems are still very limited. The objective of this book is to review the state of the art chaos control methods for some common nonlinear circuits and systems, such as those listed in the above, and stimulate further research and development in chaos control for nonlinear circuits and systems. This book consists of three parts. The first part of the book consists of reviews on general chaos control methods. In particular, a time-delayed approach written by H. Huang and G. Feng is reviewed in Chapter 1. A master slave synchronization problem for chaotic Lur’e systems is considered. A delay independent and delay dependent synchronization criteria are derived based on the H performance. The design of the time delayed feedback controller can be accomplished by means of the feasibility of linear matrix inequalities. In Chapter 2, a fuzzy model based approach written by H.K. Lam and F.H.F. Leung is reviewed. The synchronization of chaotic systems subject to parameter uncertainties is considered. A chaotic system is first represented by the fuzzy model. A switching controller is then employed to synchronize the systems. The stability conditions in terms of linear matrix inequalities are derived based on the Lyapunov stability theory. The tracking performance and parameter design of the controller are formulated as a generalized eigenvalue minimization problem which is solved numerically via some convex programming techniques. In Chapter 3, a sliding mode control approach written by Y. Feng and X. Yu is reviewed. Three kinds of sliding mode control methods, traditional sliding mode control, terminal sliding mode control and non-singular terminal sliding mode control, are employed for the control of a chaotic system to realize two different control objectives, namely to force the system states to converge to zero or to track desired trajectories. Observer based chaos synchronizations for chaotic systems with single nonlinearity and multi-nonlinearities are also presented. In Chapter 4, an optimal control approach written by C.Z. Wu, C.M. Liu, K.L. Teo and Q.X. Shao is reviewed. Systems with nonparametric regression with jump points are considered. The rough locations of all the possible jump points are identified using existing kernel methods. A smooth spline function is used to approximate each segment of the regression function. A time scaling transformation is derived so as to map the undecided jump points to fixed points. The approximation problem is formulated as an optimization problem and solved via existing optimization tools. The second part of the book consists of reviews on general chaos controls for continuous-time systems. In particular, chaos controls for Chua’s circuits written by L.A.B. Tôrres, L.A. Aguirre, R.M. Palhares and E.M.A.M. Mendes are discussed in Chapter 5. An inductorless Chua’s circuit realization is presented, as well as some practical issues, such as data analysis, mathematical modelling and dynamical characterization, are discussed. The tradeoff among the control objective, the control energy and the model complexity is derived. In Chapter 6, chaos controls for pulse width modulation current mode single phase H-bridge inverters written by B. Robert, M. Feki and H.H.C. Iu are discussed. A time delayed feedback controller is used in conjunction with the proportional controller in its simple form as well as in its extended form to stabilize the desired periodic orbit for larger values of the proportional controller gain. This method is very robust and easy to implement. In Chapter 7, chaos controls for epileptiform bursting in the brain written by M.W. Slutzky, P. Cvitanovic and D.J. Mogul are discussed. Chaos analysis and chaos control algorithms for manipulating the seizure like behaviour in a brain slice model are discussed. The techniques provide a nonlinear control pathway for terminating or potentially preventing epileptic seizures in the whole brain. The third part of the book consists of reviews on general chaos controls for discrete-time systems. In particular, chaos controls for phase lock loops written by A.M. Harb and B.A. Harb are discussed in Chapter 8. A nonlinear controller based on the theory of backstepping is designed so that the phase lock loops will not be out of lock. Also, the phase lock loops will not exhibit Hopf bifurcation and chaotic behaviours. In Chapter 9, chaos controls for sigma delta modulators written by B.W.K. Ling, C.Y.F. Ho and J.D. Reiss are discussed. A fuzzy impulsive control approach is employed for the control of the sigma delta modulators. The local stability criterion and the condition for the occurrence of limit cycle behaviours are derived. Based on the derived conditions, a fuzzy impulsive control law is formulated so that the occurrence of the limit cycle behaviours, the effect of the audio clicks and the distance between the state vectors and an invariant set are minimized supposing that the invariant set is nonempty. The state vectors can be bounded within any arbitrary nonempty region no matter what the input step size, the initial condition and the filter parameters are. The editors are much indebted to the editor of the World Scientific Series on Nonlinear Science, Prof. Leon Chua, and to Senior Editor Miss Lakshmi Narayan for their help and congenial processing of the edition

Centralized moving-horizon estimation for a class of nonlinear dynamical complex networks under event-triggered transmission scheme

Data availability statement: The data that support the findings of this study are available from the corresponding author upon reasonable request.This article is concerned with the problem of event-triggered centralized moving-horizon state estimation for a class of nonlinear dynamical complex networks. An event-triggered scheme is employed to reduce unnecessary data transmissions between sensors and estimators, where the signal is transmitted only when certain condition is violated. By treating sector-bounded nonlinearities as certain sector-bounded uncertainties, the addressed centralized moving-horizon estimation problem is transformed into a regularized robust least-squares problem that can be effectively solved via existing convex optimization algorithms. Moreover, a sufficient condition is derived to guarantee the exponentially ultimate boundedness of the estimation error, and an upper bound of the estimation error is also presented. Finally, a numerical example is provided to demonstrate the feasibility and efficiency of the proposed estimator design method.National Natural Science Foundation of China. Grant Numbers: 61873148, 61933007, 62033008, 62073339, 62173343; Natural Science Foundation of Shandong Province of China. Grant Number: ZR2020YQ49; AHPU Youth Top-notch Talent Support Program of China. Grant Number: 2018BJRC009; Natural Science Foundation of Anhui Province of China. Grant Number: 2108085MA07; China Postdoctoral Science Foundation. Grant Number: 2018T110702; Postdoctoral Special Innovation Foundation of Shandong Province of China. Grant Number: 201701015; Royal Society of the UK; Alexander von Humboldt Foundation of Germany