An Alternating Direction Method of Multipliers for Constrained Joint Diagonalization by Congruence (Invited Paper)

International audienceIn this paper, we address the problem of joint diagonalization by congruence (i.e. the canonical polyadic decomposition of semi-symmetric 3rd order tensors) subject to arbitrary convex constraints. Sufficient conditions for the existence of a solution are given. An efficient algorithm based on the Alternating Direction Method of Multipliers (ADMM) is then designed. ADMM provides an elegant approach for handling the additional constraint terms, while taking advantage of the structure of the objective function. Numerical tests on simulated matrices show the benefits of the proposed method for low signal to noise ratios. Simulations in the context of nuclear magnetic resonance spectroscopy are also provided

Exploring sparsity, self-similarity, and low rank approximation in action recognition, motion retrieval, and action spotting

This thesis consists of 4 major parts. In the first part (Chapters 1-2), we introduce the overview, motivation, and contribution of our works, and extensively survey the current literature for 6 related topics. In the second part (Chapters 3-7), we explore the concept of Self-Similarity in two challenging scenarios, namely, the Action Recognition and the Motion Retrieval. We build three-dimensional volume representations for both scenarios, and devise effective techniques that can produce compact representations encoding the internal dynamics of data. In the third part (Chapter 8), we explore the challenging action spotting problem, and propose a feature-independent unsupervised framework that is effective in spotting action under various real situations, even under heavily perturbed conditions. The final part (Chapters 9) is dedicated to conclusions and future works. For action recognition, we introduce a generic method that does not depend on one particular type of input feature vector. We make three main contributions: (i) We introduce the concept of Joint Self-Similarity Volume (Joint SSV) for modeling dynamical systems, and show that by using a new optimized rank-1 tensor approximation of Joint SSV one can obtain compact low-dimensional descriptors that very accurately preserve the dynamics of the original system, e.g. an action video sequence; (ii) The descriptor vectors derived from the optimized rank-1 approximation make it possible to recognize actions without explicitly aligning the action sequences of varying speed of execution or difference frame rates; (iii) The method is generic and can be applied using different low-level features such as silhouettes, histogram of oriented gradients (HOG), etc. Hence, it does not necessarily require explicit tracking of features in the space-time volume. Our experimental results on five public datasets demonstrate that our method produces very good results and outperforms many baseline methods. For action recognition for incomplete videos, we determine whether incomplete videos that are often discarded carry useful information for action recognition, and if so, how one can represent such mixed collection of video data (complete versus incomplete, and labeled versus unlabeled) in a unified manner. We propose a novel framework to handle incomplete videos in action classification, and make three main contributions: (i) We cast the action classification problem for a mixture of complete and incomplete data as a semi-supervised learning problem of labeled and unlabeled data. (ii) We introduce a two-step approach to convert the input mixed data into a uniform compact representation. (iii) Exhaustively scrutinizing 280 configurations, we experimentally show on our two created benchmarks that, even when the videos are extremely sparse and incomplete, it is still possible to recover useful information from them, and classify unknown actions by a graph based semi-supervised learning framework. For motion retrieval, we present a framework that allows for a flexible and an efficient retrieval of motion capture data in huge databases. The method first converts an action sequence into a self-similarity matrix (SSM), which is based on the notion of self-similarity. This conversion of the motion sequences into compact and low-rank subspace representations greatly reduces the spatiotemporal dimensionality of the sequences. The SSMs are then used to construct order-3 tensors, and we propose a low-rank decomposition scheme that allows for converting the motion sequence volumes into compact lower dimensional representations, without losing the nonlinear dynamics of the motion manifold. Thus, unlike existing linear dimensionality reduction methods that distort the motion manifold and lose very critical and discriminative components, the proposed method performs well, even when inter-class differences are small or intra-class differences are large. In addition, the method allows for an efficient retrieval and does not require the time-alignment of the motion sequences. We evaluate the performance of our retrieval framework on the CMU mocap dataset under two experimental settings, both demonstrating very good retrieval rates. For action spotting, our framework does not depend on any specific feature (e.g. HOG/HOF, STIP, silhouette, bag-of-words, etc.), and requires no human localization, segmentation, or framewise tracking. This is achieved by treating the problem holistically as that of extracting the internal dynamics of video cuboids by modeling them in their natural form as multilinear tensors. To extract their internal dynamics, we devised a novel Two-Phase Decomposition (TP-Decomp) of a tensor that generates very compact and discriminative representations that are robust to even heavily perturbed data. Technically, a Rank-based Tensor Core Pyramid (Rank-TCP) descriptor is generated by combining multiple tensor cores under multiple ranks, allowing to represent video cuboids in a hierarchical tensor pyramid. The problem then reduces to a template matching problem, which is solved efficiently by using two boosting strategies: (i) to reduce the search space, we filter the dense trajectory cloud extracted from the target video; (ii) to boost the matching speed, we perform matching in an iterative coarse-to-fine manner. Experiments on 5 benchmarks show that our method outperforms current state-of-the-art under various challenging conditions. We also created a challenging dataset called Heavily Perturbed Video Arrays (HPVA) to validate the robustness of our framework under heavily perturbed situations

Advanced signal processing concepts for multi-dimensional communication systems

Die weit verbreitete Nutzung von mobilem Internet und intelligenten Anwendungen hat zu einem explosionsartigen Anstieg des mobilen Datenverkehrs geführt. Mit dem Aufstieg von intelligenten Häusern, intelligenten Gebäuden und intelligenten Städten wächst diese Nachfrage ständig, da zukünftige Kommunikationssysteme die Integration mehrerer Netzwerke erfordern, die verschiedene Sektoren, Domänen und Anwendungen bedienen, wie Multimedia, virtuelle oder erweiterte Realität, Machine-to-Machine (M2M) -Kommunikation / Internet of Things (IoT), Automobilanwendungen und vieles mehr. Daher werden die Kommunikationssysteme zukünftig nicht nur eine drahtlose Verbindung über Gbps bereitstellen müssen, sondern auch andere Anforderungen erfüllen müssen, wie z. B. eine niedrige Latenzzeit und eine massive Maschinentyp-Konnektivität, während die Dienstqualität sichergestellt wird. Ohne bedeutende technologische Fortschritte zur Erhöhung der Systemkapazität wird die bestehende Telekommunikationsinfrastruktur diese mehrdimensionalen Anforderungen nicht unterstützen können. Dies stellt eine wichtige Forderung nach geeigneten Wellenformen und Signalverarbeitungslösungen mit verbesserten spektralen Eigenschaften und erhöhter Flexibilität dar. Aus der Spektrumsperspektive werden zukünftige drahtlose Netzwerke erforderlich sein, um mehrere Funkbänder auszunutzen, wie zum Beispiel niedrigere Frequenzbänder (typischerweise mit Frequenzen unter 10 GHz), mm-Wellenbänder (einige hundert GHz höchstens) und THz-Bänder. Viele alternative Technologien wie Optical Wireless Communication (OWC), dynamische Funksysteme und zellulares Radar sollten ebenfalls untersucht werden, um ihr wahres Potenzial abzuschätzen. Insbesondere bietet OWC ein großes, aber noch nicht genutztes optisches Band im sichtbaren Spektrum, das Licht als Mittel zur Informationsübertragung nutzt. Daher können zukünftige Kommunikationssysteme als zusammengesetzte Hybridnetzwerke angesehen werden, die aus einer Anzahl von verschiedenen drahtlosen Netzwerken bestehen, die auf Funk und optischem Zugang basieren. Auf der anderen Seite ist es eine große Herausforderung, fortschrittliche Signalverarbeitungslösungen für mehrere Bereiche von Kommunikationssystemen zu entwickeln. Diese Arbeit trägt zu diesem Ziel bei, indem sie Methoden für die Suche nach effizienten algebraischen Lösungen für verschiedene Anwendungen der digitalen Mehrkanal-Signalverarbeitung demonstriert. Insbesondere tragen wir zu drei verschiedenen Anwendungsgebieten bei, d.h. Wellenformen, optischen drahtlosen Systemen und mehrdimensionaler Signalverarbeitung. Gegenwärtig ist das Cyclic Prefix Orthogonal Frequency Division Multiplexing (CP-OFDM) die weit verbreitete Multitragetechnik für die meisten Kommunikationssysteme. Um jedoch die CP-OFDM-Nachteile in Bezug auf eine schlechte spektrale Eingrenzung, Robustheit in hoch asynchronen Umgebungen und Unflexibilität der Parameterwahl zu überwinden, wurden viele alternative Wellenformen vorgeschlagen. Solche Mehrfachträgerwellenformen umfassen einen Filter bank Multicarrier (FBMC), ein Generalized Frequency Division Multiplexing (GFDM), einen Universal Filter Multicarrier (UFMC) und ein Unique Word Orthogonal Orthogonal Frequency Division Multiplexing (UW-OFDM). Diese neuen Luftschnittstellenschemata verwenden verschiedene Ansätze, um einige der inhärenten Mängel bei CP-OFDM zu überwinden. Einige dieser Wellenformen wurden gut untersucht, während andere sich noch in den Kinderschuhen befinden. Insbesondere die Integration von Multiple-Input-Multiple-Output (MIMO) -Konzepten mit UW-OFDM und UFMC befindet sich noch in einem frühen Forschungsstadium. Daher schlagen wir im ersten Teil dieser Arbeit neuartige lineare und sukzessive Interferenzunterdrückungstechniken für MIMO UW-OFDM-Systeme vor. Das Design dieser Techniken zielt darauf ab, Empfänger mit einer geringen Rechenkomplexität zu erhalten. Ein weiterer Schwerpunkt ist die Anwendbarkeit von Space-Time Block Codes (STBCs) auf UW-OFDM und UFMC-Wellenformen. Zu diesem Zweck stellen wir neue Techniken zusammen mit Detektionsverfahren vor. Wir vergleichen auch die Leistung dieser Wellenformen mit unseren vorgeschlagenen Techniken mit den anderen Wellenformen des Standes der Technik, die in der Literatur vorgeschlagen wurden. Wir zeigen, dass raumzeitblockierte UW-OFDM-Systeme mit den vorgeschlagenen Methoden nicht nur andere Wellenformen signifikant übertreffen, sondern auch zu Empfängern mit geringer Rechnerkomplexität führen. Der zweite Anwendungsbereich umfasst optische Systeme im sichtbaren Band (390-700 nm), die in Plastic Optical Fibers (POFs), Multimode-Fasern oder OWC-Systemen wie der Kommunikation über Visible Light Communication (VLC) verwendet werden können. VLC kann Lösungen für eine Reihe von Anwendungen anbieten, einschließlich drahtloser lokaler, persönlicher und Körperbereichsnetzwerke (WLAN, WPAN und WBANs), Innenlokalisierung und -navigation, Fahrzeugnetze, U-Bahn- und Unterwassernetze und bietet eine Reihe von Datenraten von wenigen Mbps zu 10 Gbps. VLC nutzt voll sichtbare Light Emitting Diodes (LEDs) für den doppelten Zweck der Beleuchtung und Datenkommunikation bei sehr hohen Geschwindigkeiten. Daher verwenden solche Systeme Intensitätsmodulation und Direct Detection (IM / DD), wodurch gefordert wird, dass das Sendesignal reellwertig und positiv sein sollte. Dies impliziert auch, dass die herkömmlichen Wellenformen, die für die Radio Frequency (RF) Kommunikation ausgelegt sind, nicht direkt verwendet werden können. Zum Beispiel muss eine hermetische Symmetrie auf das CP-OFDM angewendet werden, um ein reellwertiges Signal zu erhalten (oft als Discrete Multitone Transmission (DMT) bezeichnet), das im Gegenzug die Bandbreiteneffizienz verringert. Darüber hinaus begrenzt die LED / LED-Treiberkombination die elektrische Bandbreite. Alle diese Faktoren erfordern die Verwendung spektral effizienter Übertragungsverfahren zusammen mit robusten Entzerrungsschemata, um hohe Datenraten zu erzielen. Deshalb schlagen wir im zweiten Teil der Arbeit Übertragungsverfahren vor, die für solche optischen Systeme am besten geeignet sind. Insbesondere demonstrieren wir die Leistung der PAM-Blockübertragung mit Frequenzbereichsausgleich. Wir zeigen, dass dieses Schema nicht nur leistungsstärker ist, sondern auch alle modernen Verfahren wie CP-DMT-Schemata übertrifft. Wir schlagen auch neue UW-DMT-Schemata vor, die vom UW-OFDM-Konzept abgeleitet sind. Diese Schemata zeigen auch ein sehr überlegenes Bitfehlerverhältnis (BER) -Performance gegenüber den herkömmlichen CP-DMT-Schemata. Der dritte Anwendungsbereich konzentriert sich auf mehrdimensionale Signalverarbeitungstechniken. Bei der Verwendung von MIMO, STBCs, Mehrbenutzerverarbeitung und Mehrträgerwellenformen bei der drahtlosen Kommunikation ist das empfangene Signal mehrdimensional und kann eine multilineare Struktur aufweisen. In diesem Zusammenhang können Signalverarbeitungstechniken, die auf einem Tensor-Modell basieren, gleichzeitig von mehreren Formen von Diversität profitieren, um Mehrbenutzer-Signaltrennung / -entzerrung und Kanalschätzung durchzuführen. Dieser Vorteil ist eine direkte Konsequenz der Eigenschaft der wesentlichen Eindeutigkeit, die für matrixbasierte Ansätze nicht verfügbar ist. Tensor-Zerlegung wie die Higher Order Singular Value Decomposition (HOSVD) und die Canonical Polyadic Decomposition (CPD) werden weithin zur Durchführung dieser Aufgaben empfohlen. Die Leistung dieser Techniken wird oft mit zeitraubenden Monte-Carlo-Versuchen bewertet. Im letzten Teil der Arbeit führen wir eine Störungsanalyse erster Ordnung dieser Tensor-Zerlegungsmethoden durch. Insbesondere führen wir eine analytische Performanceanalyse des Semi-algebraischen Frameworks für approximative Canonical polyadic decompositions Simultaneous matrix diagonalizations (SECSI) durch. Das SECSI-Framework ist ein effizientes Werkzeug zur Berechnung der CPD eines rauscharmen Tensor mit niedrigem Rang. Darüber hinaus werden die erhaltenen analytischen Ausdrücke in Bezug auf die Momente zweiter Ordnung des Rauschens formuliert, so dass abgesehen von einem Mittelwert von Null keine Annahmen über die Rauschstatistik erforderlich sind. Wir zeigen, dass die abgeleiteten analytischen Ergebnisse eine ausgezeichnete Übereinstimmung mit den Monte-Carlo-Simulationen zeigen.The widespread use of mobile internet and smart applications has led to an explosive growth in mobile data traffic. With the rise of smart homes, smart buildings, and smart cities, this demand is ever growing since future communication systems will require the integration of multiple networks serving diverse sectors, domains and applications, such as multimedia, virtual or augmented reality, machine-to-machine (M2M) communication / the Internet of things (IoT), automotive applications, and many more. Therefore, in the future, the communication systems will not only be required to provide Gbps wireless connectivity but also fulfil other requirements such as low latency and massive machine type connectivity while ensuring the quality of service. Without significant technological advances to increase the system capacity, the existing telecommunications infrastructure will be unable to support these multi-dimensional requirements. This poses an important demand for suitable waveforms with improved spectral characteristics and signal processing solutions with an increased flexibility. Moreover, future wireless networks will be required to exploit several frequency bands, such as lower frequency bands (typically with frequencies below 10 GHz), mm-wave bands (few hundred GHz at the most), and THz bands. Many alternative technologies such as optical wireless communication (OWC), dynamic radio systems, and cellular radar should also be investigated to assess their true potential. Especially, OWC offers large but yet unexploited optical band in the visible spectrum that uses light as a means to carry information. Therefore, future communication systems can be seen as composite hybrid networks that consist of a number of different wireless networks based on radio and optical access. On the other hand, it poses a significant challenge to come up with advanced signal processing solutions in multiple areas of communication systems. This thesis contributes to this goal by demonstrating methods for finding efficient algebraic solutions to various applications of multi-channel digital signal processing. In particular, we contribute to three different scientific fields, i.e., waveforms, optical wireless systems, and multi-dimensional signal processing. Currently, cyclic prefix orthogonal frequency division multiplexing (CP-OFDM) is the widely adopted multicarrier technique for most of the communication systems. However, to overcome the CP-OFDM demerits in terms of poor spectral containment, poor robustness in highly asynchronous environments, and inflexibility of parameter choice, and many alternative waveforms have been proposed. Such multicarrier waveforms include filter bank multicarrier (FBMC), generalized frequency division multiplexing (GFDM), universal filter multicarrier (UFMC), and unique word orthogonal frequency division multiplexing (UW-OFDM). These new air interface schemes take different approaches to overcome some of the inherent deficiencies in CP-OFDM. Some of these waveforms have been well investigated while others are still in its infancy. Specifically, the integration of multiple-input multiple-output (MIMO) concepts with UW-OFDM and UFMC is still at an early stage of research. Therefore, in the first part of this thesis, we propose novel linear and successive interference cancellation techniques for MIMO UW-OFDM systems. The design of these techniques is aimed to result in receivers with a low computational complexity. Another focus area is the applicability of space-time block codes (STBCs) to UW-OFDM and UFMC waveforms. For this purpose, we present novel techniques along with detection procedures. We also compare the performance of these waveforms with our proposed techniques to the other state-of-the-art waveforms that has been proposed in the literature. We demonstrate that space-time block coded UW-OFDM systems with the proposed methods not only outperform other waveforms significantly but also results in receivers with a low computational complexity. The second application area comprises of optical systems in the visible band (390-700 nm) that can be utilized in plastic optical fibers (POFs), multimode fibers or OWC systems such as visible light communication (VLC). VLC can provide solutions for a number of applications including wireless local, personal, and body area networks (WLAN, WPAN, and WBANs), indoor localization and navigation, vehicular networks, underground and underwater networks, offering a range of data rates from a few Mbps to 10 Gbps. VLC takes full advantage of visible light emitting diodes (LEDs) for the dual purpose of illumination and data communications at very high speeds. Because of the incoherent nature of the LED sources, such systems employ intensity modulation and direct detection (IM/DD), thus demanding that the transmit signal should be real-valued and positive. This also implies that the conventional waveforms designed for the radio frequency (RF) communication cannot be directly used. For example, a Hermitian symmetry has to be applied to the CP-OFDM spectrum to obtain a real-valued signal (often referred to as discrete multitone transmission (DMT)) that in return reduces the bandwidth efficiency. Moreover, the LED/LED driver combination limits the electrical bandwidth. All these factors require the use of spectrally efficient transmission schemes along with robust equalization schemes to achieve high data rates. Therefore, in the second part of the thesis, we propose transmission schemes that are best suited for such optical systems. Specifically, we demonstrate the performance of PAM block transmission with frequency domain equalization. We show that this scheme is not only more power efficient but also outperforms all of the state-of-the-art schemes such as CP-DMT schemes. We also propose novel UW-DMT schemes that are derived from the UW-OFDM concept. These schemes also show a much superior bit error ratio (BER) performance over the conventional CP-DMT schemes. The third application area focuses on multi-dimensional signal processing techniques. With the use of MIMO, STBCs, multi-user processing, and multicarrier waveforms in wireless communications, the received signal is multidimensional in nature and may exhibit a multilinear structure. In this context, signal processing techniques based on a tensor model can simultaneously benefit from multiple forms of diversity to perform multi-user signal separation/equalization and channel estimation. This advantage is a direct consequence of the essential uniqueness property that is not available for matrix based approaches. Tensor decompositions such as the higher order singular value decomposition (HOSVD) and the canonical polyadic decomposition (CPD) are widely recommended for performing these tasks. The performance of these techniques is often evaluated using time consuming Monte-Carlo trials. In the last part of the thesis, we perform a first-order perturbation analysis of the truncated HOSVD and the Semi-algebraic framework for approximate Canonical polyadic decompositions via Simultaneous matrix diagonalizations (SECSI). The SECSI framework is an efficient tool for the computation of the approximate CPD of a low-rank noise corrupted tensor. Especially, the SECSI framework shows a much improved performance and comparatively low-complexity as compared to the conventional algorithms such as alternative least squares (ALS). Moreover, it also facilitates the implementation on a parallel hardware architecture. The obtained analytical expressions for both algorithms are formulated in terms of the second-order moments of the noise, such that apart from a zero-mean, no assumptions on the noise statistics are required. We demonstrate that the derived analytical results exhibit an excellent match to the Monte-Carlo simulations

Numerical construction of Green’s functions in high dimensional elliptic problems with variable coefficients and analysis of renewable energy data via sparse and separable approximations

This thesis consists of two parts. In Part I, we describe an algorithm for approximating the Green\u27s function for elliptic problems with variable coefficients in arbitrary dimension. The basis for our approach is the separated representation, which appears as a way of approximating functions of many variables by sums of products of univariate functions. While the differential operator we wish to invert is typically ill-conditioned, its conditioning may be improved by first applying the Green\u27s function for the constant coefficient problem. This function may be computed either numerically or, in some case, analytically in a separated format. The variable coefficient Green\u27s function is then computed using a quadratically convergent iteration on the preconditioned operator, with sparsity maintained via representation in a wavelet basis. Of particular interest is that the method scales linearly in the number of dimensions, a feature that very desirable in high dimensional problems in which the curse of dimensionality must be reckoned with. As a corollary to this work, we described a randomized algorithm for maintaining low separation rank of the functions used in the construction of the Green\u27s function. For certain functions of practical interest, one can avoid the cost of using standard methods such as alternating least squares (ALS) to reduce the separation rank. Instead, terms from the separated representation may be selected using a randomized approach based on matrix skeletonization and the interpolative decomposition. The use of random projections can greatly reduce the cost of rank reduction, as well as calculation of the Frobenius norm and term-wise Gram matrices. In Part II of the thesis, we highlight three practical applications of sparse and separable approximations to the analysis of renewable energy data. In the first application, error estimates gleaned from repeated measurements are incorporated into sparse regression algorithms (LASSO and the Dantzig selector) to minimize the statistical uncertainty of the resulting model. Applied to real biomass data, this approach leads to sparser regression coefficients corresponding to improved accuracy as measured by k-fold cross validation error. In the second application, a regression model based on separated representations is fit to reliability data for cadmium telluride (CdTe) thin-film solar cells. The data is inherently multi-way, and our approach avoids artificial matricization that would typically be performed for use with standard regression algorithms. Two distinct modes of degradation, corresponding to short- and long-term decrease in cell efficiency, are identified. In the third application, some theoretical properties of a popular chemometrics algorithm called orthogonal projections to latent structures (O-PLS) are derived

Generalized averaged Gaussian quadrature and applications

A simple numerical method for constructing the optimal generalized averaged Gaussian quadrature formulas will be presented. These formulas exist in many cases in which real positive GaussKronrod formulas do not exist, and can be used as an adequate alternative in order to estimate the error of a Gaussian rule. We also investigate the conditions under which the optimal averaged Gaussian quadrature formulas and their truncated variants are internal

MS FT-2-2 7 Orthogonal polynomials and quadrature: Theory, computation, and applications

Quadrature rules find many applications in science and engineering. Their analysis is a classical area of applied mathematics and continues to attract considerable attention. This seminar brings together speakers with expertise in a large variety of quadrature rules. It is the aim of the seminar to provide an overview of recent developments in the analysis of quadrature rules. The computation of error estimates and novel applications also are described