245 research outputs found
The Nested Periodic Subspaces: Extensions of Ramanujan Sums for Period Estimation
In the year 1918, the Indian mathematician Srinivasa Ramanujan proposed a set of sequences called Ramanujan Sums as bases to expand arithmetic functions in number theory. Today, exactly a 100 years later, we will show that these sequences re-emerge as exciting tools in a completely different context: For the extraction of periodic patterns in data. Combined with the state-of-the-art techniques of DSP, Ramanujan Sums can be used as the starting point for developing powerful algorithms for periodicity applications.
The primary inspiration for this thesis comes from a recent extension of Ramanujan sums to subspaces known as the Ramanujan subspaces. These subspaces were designed to span any sequence with integer periodicity, and have many interesting properties. Starting with Ramanujan subspaces, this thesis first develops an entire family of such subspace representations for periodic sequences. This family, called Nested Periodic Subspaces due to their unique structure, turns out to be the least redundant sets of subspaces that can span periodic sequences.
Three classes of new algorithms are proposed using the Nested Periodic Subspaces: dictionaries, filter banks, and eigen-space methods based on the auto-correlation matrix of the signal. It will be shown that these methods are especially advantageous to use when the data-length is short, or when the signal is a mixture of multiple hidden periods. The dictionary techniques were inspired by recent advances in sparsity based compressed sensing. Apart from the l1 norm based convex programs currently used in other applications, our dictionaries can admit l2 norm formulations that have linear and closed form solutions, even when the systems is under-determined. A new filter bank is also proposed using the Ramanujan sums. This, named the Ramanujan Filter Bank, can accurately track the instantaneous period for signals that exhibit time varying periodic nature. The filters in the Ramanujan Filter Bank have simple integer valued coefficients, and directly tile the period vs time plane, unlike classical STFT (Short Time Fourier Transform) and wavelets, which tile the time-frequency plane. The third family of techniques developed here are a generalization of the classic MUSIC (MUltiple SIgnal Classification) algorithm for periodic signals. MUSIC is one of the most popular techniques today for line spectral estimation. However, periodic signals are not just any unstructured line spectral signals. There is a nice harmonic spacing between the lines which is not exploited by plain MUSIC. We will show that one can design much more accurate adaptations of MUSIC using Nested Periodic Subspaces. Compared to prior variants of MUSIC for the periodicity problem, our approach is much faster and yields much more accurate results for signals with integer periods. This work is also the first extension of MUSIC that uses simple integer valued basis vectors instead of using traditional complex-exponentials to span the signal subspace. The advantages of the new methods are demonstrated both on simulations, as well as real world applications such as DNA micro-satellites, protein repeats and absence seizures.
Apart from practical contributions, the theory of Nested Periodic Subspaces offers answers to a number of fundamental questions that were previously unanswered. For example, what is the minimum contiguous data-length needed to be able to identify the period of a signal unambiguously? Notice that the answer we seek is a fundamental identifiability bound independent of any particular period estimation technique. Surprisingly, this basic question has never been answered before. In this thesis, we will derive precise expressions for the minimum necessary and sufficient datalengths for this question. We also extend these bounds to the context of mixtures of periodic signals. Once again, even though mixtures of periodic signals often occur in many applications, aspects such as the unique identifiability of the component periods were never rigorously analyzed before. We will present such an analysis as well.
While the above question deals with the minimum contiguous datalength required for period estimation, one may ask a slightly different question: If we are allowed to pick the samples of a signal in a non-contiguous fashion, how should we pick them so that we can estimate the period using the least number of samples? This question will be shown to be quite difficult to answer in general. In this thesis, we analyze a smaller case in this regard, namely, that of resolving between two periods. It will be shown that the analysis is quite involved even in this case, and the optimal sampling pattern takes an interesting form of sparsely located bunches. This result can also be extended to the case of multi-dimensional periodic signals.
We very briefly address multi-dimensional periodicity in this thesis. Most prior DSP literature on multi-dimensional discrete time periodic signals assumes the period to be parallelepipeds. But as shown by the artist M. C. Escher, one can tile the space using a much more diverse variety of shapes. Is it always possible to account for such other periodic shapes using the traditional notion of parallelepiped periods? An interesting analysis in this regard is presented towards the end of the thesis.</p
Towards music perception by redundancy reduction and unsupervised learning in probabilistic models
PhDThe study of music perception lies at the intersection of several disciplines: perceptual
psychology and cognitive science, musicology, psychoacoustics, and acoustical
signal processing amongst others. Developments in perceptual theory over the last
fifty years have emphasised an approach based on Shannon’s information theory and
its basis in probabilistic systems, and in particular, the idea that perceptual systems
in animals develop through a process of unsupervised learning in response to natural
sensory stimulation, whereby the emerging computational structures are well adapted
to the statistical structure of natural scenes. In turn, these ideas are being applied to
problems in music perception.
This thesis is an investigation of the principle of redundancy reduction through
unsupervised learning, as applied to representations of sound and music.
In the first part, previous work is reviewed, drawing on literature from some of the
fields mentioned above, and an argument presented in support of the idea that perception
in general and music perception in particular can indeed be accommodated within
a framework of unsupervised learning in probabilistic models.
In the second part, two related methods are applied to two different low-level representations.
Firstly, linear redundancy reduction (Independent Component Analysis)
is applied to acoustic waveforms of speech and music. Secondly, the related method of
sparse coding is applied to a spectral representation of polyphonic music, which proves
to be enough both to recognise that the individual notes are the important structural elements,
and to recover a rough transcription of the music.
Finally, the concepts of distance and similarity are considered, drawing in ideas
about noise, phase invariance, and topological maps. Some ecologically and information
theoretically motivated distance measures are suggested, and put in to practice in
a novel method, using multidimensional scaling (MDS), for visualising geometrically
the dependency structure in a distributed representation.Engineering and Physical Science Research Counci
Estimation of Radio Channel Parameters
Kurzfassung
Diese Dissertation behandelt die Schätzung der Modellparameter einer
Momentanaufnahme des Mobilfunkkanals. Das besondere Augenmerk liegt zum einen
auf der Entwicklung eines generischen Datenmodells fĂĽr den gemessenen Funkkanal,
welches für die hochauflösende Parameterschätzung geeignet ist. Der zweite
Schwerpunkt dieser Arbeit ist die Entwicklung eines robusten Parameterschätzers
fĂĽr die Bestimmung der Parameter des entworfenen Modells aus Funkkanalmessdaten.
Entsprechend dieser logischen Abfolge ist auch der Aufbau dieser Arbeit.
Im ersten Teil wird ausgehend von einem aus der Literatur bekannten
strahlenoptischen Modell eine algebraisch handhabbare Darstellung von
beobachteten Wellenausbreitungspfaden entwickelt. Das mathematische Modell
erlaubt die Beschreibung von SISO (single-input-single-output)-
Ăśbertragungssystemen, also von Systemen mit einer Sendeantenne und einer
Empfangsantenne, als auch die Beschreibung von solchen Systemen mit mehreren
Sende- und/oder Empfangsantennen. Diese Systeme werden im Allgemeinen auch als
SIMO- (single-input-multiple-output), MISO- (multiple-input-single-output) oder
MIMO-Systeme (multiple-input-multiple-output) bezeichnet. Im Gegensatz zu
bekannten Konzepten enthält das entwickelte Modell keine Restriktionen bezüglich
der modellierbaren Antennenarrayarchitekturen. Dies ist besonders wichtig in
Hinblick auf die möglichst vollständige Erfassung der räumlichen Struktur des
Funkkanals. Die Flexibilität des Modells ist eine Grundvoraussetzung für die
optimale Anpassung der Antennenstruktur an die Messaufgabe. Eine solche
angepasste Antennenarraystruktur ist zum Beispiel eine zylindrische Anordnung
von Antennenelementen. Sie ist gut geeignet für die Erfassung der räumlichen
Struktur des Funkkanals (Azimut und Elevation) in so genannten Outdoor-
Funkszenarien. Weiterhin wird im ersten Teil eine neue Komponente des
Funkkanaldatenmodells eingefĂĽhrt, welche den Beitrag verteilter (diffuser)
Streuungen zur FunkĂĽbertragung beschreibt. Die neue Modellkomponente spielt eine
Schlüsselrolle bei der Entwicklung eines robusten Parameterschätzers im
Hauptteil dieser Arbeit. Die fehlende Modellierung der verteilten Streuungen ist
eine der Hauptursachen fĂĽr die begrenzte Anwendbarkeit und die oft kritisierte
fehlende Robustheit von hochauflösenden Funkkanalparameterschätzern, die in der
Literatur etabliert sind. Das neue Datenmodell beschreibt die so genannten
dominanten Ausbreitungspfade durch eine deterministische Abbildung der
Pfadparameter auf den gemessenen Funkkanal. Der Beitrag der verteilten
Streuungen wird mit Hilfe eines zirkularen mittelwertfreien GauĂźschen Prozesses
beschrieben. Die Modellparameter der verteilten Streuungen beschreiben dabei die
Kovarianzmatrix dieses Prozesses. Basierend auf dem entwickelten Datenmodell
wird im Anschluss kurz über aktuelle Konzepte für Funkkanalmessgeräte, so
genannte Channel-Sounder, diskutiert.
Im zweiten Teil dieser Arbeit werden in erster Linie AusdrĂĽcke zur Bestimmung
der erzielbaren Messgenauigkeit eines Channel-Sounders abgeleitet. Zu diesem
Zweck wird die untere Schranke für die Varianz der geschätzten Modellparameter,
das heißt der Messwerte, bestimmt. Als Grundlage für die Varianzabschätzung wird
das aus der Parameterschätztheorie bekannte Konzept der Cramér-Rao-Schranke
angewandt. Im Rahmen der Ableitung der Cramér-Rao-Schranke werden außerdem
wichtige Gesichtspunkte für die Entwicklung eines effizienten Parameterschätzers
diskutiert.
Im dritten Teil der Arbeit wird ein Schätzer für die Bestimmung der
Ausbreitungspfadparameter nach dem Maximum-Likelihood-Prinzip entworfen. Nach
einer kurzen Übersicht über existierende Konzepte zur hochauflösenden
Funkkanalparameterschätzung wird die vorliegende Schätzaufgabe analysiert und in
Hinsicht ihres Typs klassifiziert. Unter der Voraussetzung, dass die Parameter
der verteilten Streuungen bekannt sind, lässt sich zeigen, daß sich die
Schätzung der Parameter der Ausbreitungspfade als ein nichtlineares gewichtetes
kleinstes Fehlerquadratproblem auffassen lässt. Basierend auf dieser Erkenntnis
wird ein generischer Algorithmus zur Bestimmung einer globalen Startlösung für
die Parameter eines Ausbreitungspfades vorgeschlagen. Hierbei wird von dem
Konzept der Structure-Least-Squares (SLS)-Probleme Gebrauch gemacht, um die
Komplexität des Schätzproblems zu reduzieren. Im folgenden Teil dieses
Abschnitts wird basierend auf aus der Literatur bekannten robusten numerischen
Algorithmen ein Schätzer zur genauen Bestimmung der Ausbreitungspfadparameter
abgeleitet. Im letzten Teil dieses Abschnitts wird die Anwendung
unterraumbasierter Schätzer zur Bestimmung der Ausbreitungspfadparameter
diskutiert. Es wird ein speichereffizienter Algorithmus zur Signalraumschätzung
entwickelt. Dieser Algorithmus ist eine Grundvoraussetzung fĂĽr die Anwendung von
mehrdimensionalen Parameterschätzern wie zum Beispiel des R-D unitary ESPRIT
(Estimation of Signal Parameters via Rotational Invariance Techniques) zur
Bestimmung von Funkkanalparametern aus MIMO-Funkkanalmessungen. Traditionelle
Verfahren zur Signalraumschätzung sind hier im Allgemeinen nicht anwendbar, da
sie einen zu groĂźen Speicheraufwand erfordern. AuĂźerdem wird in diesem Teil
gezeigt, dass ESPRIT-Algorithmen auch zur Parameterschätzung von Daten mit so
genannter versteckter Rotations-Invarianzstruktur eingesetzt werden können. Als
Beispiel wird ein ESPRIT-basierter Algorithmus zur Richtungsschätzung in
Verbindung mit multibeam-Antennenarrays (CUBA) abgeleitet.
Im letzten Teil dieser Arbeit wird ein Maximum-Likelihood-Schätzer für die neue
Komponente des Funkkanals, welche die verteilten Streuungen beschreibt,
entworfen. Ausgehend vom Konzept des iterativen Maximum-Likelihood-Schätzers
wird ein Algorithmus entwickelt, der hinreichend geringe numerische Komplexität
besitzt, so dass er praktisch anwendbar ist. In erster Linie wird dabei von der
Toeplitzstruktur der zu schätzenden Kovarianzmatrix Gebrauch gemacht. Aufbauend
auf dem Schätzer für die Parameter der Ausbreitungspfade und dem Schätzer für
die Parameter der verteilten Streuungen wird ein Maximum-Likelihood-Schätzer
entwickelt (RIMAX), der alle Parameter des in Teil I entwickelten Modells der
Funkanalmessung im Verbund schätzt. Neben den geschätzten Parametern des
Datenmodells liefert der Schätzer zusätzlich Zuverlässigkeitsinformationen.
Diese werden unter anderem zur Bestimmung der Modellordnung, das heiĂźt zur
Bestimmung der Anzahl der dominanten Ausbreitungspfade, herangezogen. AuĂźerdem
stellen die Zuverlässigkeitsinformationen aber auch ein wichtiges Schätzergebnis
dar. Die Zuverlässigkeitsinformationen machen die weitere Verarbeitung und
Wertung der Messergebnisse möglich.The theme of this thesis is the estimation of model parameters of a radio channel snapshot. The main focus was the development of a general data model for the measured radio channel, suitable for both high resolution channel parameter estimation on the one hand, and the development of a robust parameter estimator
for the parameters of the designed parametric radio channel model, in line with this logical work flow is this thesis.
In the first part of this work an algebraic representation of observed
propagation paths is developed using a ray-optical model known from literature. The algebraic framework is suitable for the description of SISO (single-input-single-output) radio transmission systems. A SISO system uses one antenna as the transmitter (Tx) and one antenna as the receiver (Rx). The derived expression for the propagation paths is also suitable to describe SIMO (single-input-multiple-output), MISO (multiple-input-single-output), and MIMO (multiple-input-multiple-output) radio channel measurements. In contrast to other models used for high resolution channel parameter estimation the derived model makes no
restriction regarding the structure of the antenna array used throughout the measurement. This is important since the ultimate goal in radio channel sounding is the complete description of the spatial (angular) structure of the radio channel at Tx and Rx. The flexibility of the data model is a prerequisite for the optimisation of the antenna array structure with respect to the measurement
task. Such an optimised antenna structure is a stacked uniform circular beam array, i.e., a cylindrical arrangement of antenna elements. This antenna array configuration is well suited for the measurement of the spatial structure of the radio channel at Tx and/or Rx in outdoor-scenarios. Furthermore, a new component
of the radio channel model is introduced in the first part of this work. It describes the contribution of distributed (diffuse) scattering to the radio transmission. The new component is key for the development of a robust radio channel parameter estimator, which is derived in the main part of this work. The ignorance of the contribution of distributed scattering to radio propagation is one of the main reasons why high-resolution radio channel parameter estimators fail in practice. Since the underlying data model is wrong the estimators produce erroneous results. The improved model describes the so called dominant propagation paths by a deterministic mapping of the propagation path parameters
to the channel observation. The contribution of the distributed scattering is modelled as a zero-mean circular Gaussian process. The parameters of the distributed scattering process determine the structure of the covariance matrix of this process. Based on this data model current concepts for radio channel sounding devices are discussed.
In the second part of this work expressions for the accuracy achievable by a radio channel sounder are derived. To this end the lower bound on the variance of the measurements i.e. the parameter estimates is derived. As a basis for this evaluation the concept of the Cramér-Rao lower bound is employed. On the way to
the Cramér-Rao lower bound for all channel model parameters, important issues for the development of an appropriate parameter estimator are discussed. Among other things the coupling of model parameters is also discussed.
In the third part of this thesis, an estimator, for the propagation path parameters is derived. For the estimator the 'maximum-likelihood' approach is employed. After a short overview of existing high-resolution channel parameter estimators the estimation problem is classified. It is shown, that the estimation of the parameters of the propagation paths can be understood as a
nonlinear weighted least squares problem, provided the parameters of the distributed scattering process are known. Based on this observation a general algorithm for the estimation of raw parameters for the observed propagation paths is developed. The algorithm uses the concept of structured-least-squares (SLS) and compressed maximum likelihood to reduce the numerical complexity of the estimation problem. A robust estimator for the precise estimation of the propagation path parameters is derived. The estimator is based on concepts well known from nonlinear local optimisation theory. In the last part of this chapter the application of subspace based parameter estimation algorithms for path
parameter estimation is discussed. A memory efficient estimator for the signal subspace needed by, e.g., R-D unitary ESPRIT is derived. This algorithm is a prerequisite for the application of signal subspace based algorithms to MIMO-channel sounding measurements. Standard algorithms for signal subspace estimation (economy size SVD, singular value decomposition) are not suitable
since they require an amount of memory which is too large. Furthermore, it is shown that ESPRIT (Estimation of Signal Parameters via Rotational Invariance Techniques) based algorithms can also be employed for parameter estimation from data having hidden rotation invariance structure. As an example an ESPRIT
algorithm for angle estimation using circular uniform beam arrays (circular multi-beam antennas) is derived.
In the final part of this work a maximum likelihood estimator for the new component of the channel model is developed. Starting with the concept of iterative maximum likelihood estimation, an algorithm is developed having a low computational complexity. The low complexity of the algorithm is achieved by exploiting the Toeplitz-structure of the covariance matrix to estimate. Using
the estimator for the (concentrated, dominant, specular-alike) propagation paths and the parametric estimator for the covariance matrix of the process describing the distributed diffuse scattering a joint estimator for all channel parameter is derived (RIMAX). The estimator is a 'maximum likelihood' estimator and uses the genuine SAGE concept to reduce the computational complexity. The estimator provides additional information about the reliability of the estimated channel parameters. This reliability information is used to determine an appropriate model for the observation. Furthermore, the reliability information i.e. the estimate of the covariance matrix of all parameter estimates is also an important parameter estimation result. This information is a prerequisite for further processing and evaluation of the measured channel parameters
Introduction to frames
This survey gives an introduction to redundant signal representations called frames. These representations have recently emerged as yet another powerful tool in the signal processing toolbox and have become popular through use in numerous applications. Our aim is to familiarize a general audience with the area, while at the same time giving a snapshot of the current state-of-the-art
Recommended from our members
Signal separation of musical instruments: simulation-based methods for musical signal decomposition and transcription
This thesis presents techniques for the modelling of musical signals, with particular regard to monophonic and polyphonic pitch estimation. Musical signals are modelled as a set of notes, each comprising of a set of harmonically-related sinusoids. An hierarchical model is presented that is very general and applicable to any signal that can be decomposed as the sum of basis functions. Parameter estimation is posed within a Bayesian framework, allowing for the incorporation of prior information about model parameters. The resulting posterior distribution is of variable dimension and so reversible jump MCMC simulation techniques are employed for the parameter estimation task. The extension of the model to time-varying signals with high posterior correlations between model parameters is described. The parameters and hyperparameters of several frames of data are estimated jointly to achieve a more robust detection. A general model for the description of time-varying homogeneous and heterogeneous multiple component signals is developed, and then applied to the analysis of musical signals. The importance of high level musical and perceptual psychological knowledge in the formulation of the model is highlighted, and attention is drawn to the limitation of pure signal processing techniques for dealing with musical signals. Gestalt psychological grouping principles motivate the hierarchical signal model, and component identifiability is considered in terms of perceptual streaming where each component establishes its own context. A major emphasis of this thesis is the practical application of MCMC techniques, which are generally deemed to be too slow for many applications. Through the design of efficient transition kernels highly optimised for harmonic models, and by careful choice of assumptions and approximations, implementations approaching the order of realtime are viable.Engineering and Physical Sciences Research Counci
Latent variable regression and applications to planetary seismic instrumentation
The work presented in this thesis is framed by the concept of latent variables, a modern data analytics approach. A latent variable represents an extracted component from a dataset which is not directly measured.
The concept is first applied to combat the problem of ill-posed regression through the promising method of partial least squares (PLS). In this context the latent variables within a data matrix are extracted through an iterative algorithm based on cross-covariance as an optimisation criterion. This work first extends the PLS algorithm, using adaptive and recursive techniques, for online, non-stationary data applications. The standard PLS algorithm is further generalised for complex-, quaternion- and tensor-valued data. In doing so it is shown that the multidimensional algebras facilitate physically meaningful representations, demonstrated through smart-grid frequency estimation and image-classification tasks.
The second part of the thesis uses this knowledge to inform a performance analysis of the MEMS microseismometer implemented for the InSight mission to Mars. This is given in terms of the sensor's intrinsic self-noise, the estimation of which is achieved from experimental data with a colocated instrument. The standard coherence and proposed delta noise estimators are analysed with respect to practical issues. The implementation of algorithms for the alignment, calibration and post-processing of the data then enabled a definitive self-noise estimate, validated from data acquired in ultra-quiet, deep-space environment.
A method for the decorrelation of the microseismometer's output from its thermal response is proposed. To do so a novel sensor fusion approach based on the Kalman filter is developed for a full-band transfer-function correction, in contrast to the traditional ill-posed frequency division method. This algorithm was applied to experimental data which determined the thermal model coefficients while validating the sensor's performance at tidal frequencies 1E-5Hz and in extreme environments at -65C.
This thesis, therefore, provides a definitive view of the latent variables perspective. This is achieved through the general algorithms developed for regression with multidimensional data and the bespoke application to seismic instrumentation.Open Acces
Directional multiresolution image representations
Efficient representation of visual information lies at the foundation of many image processing tasks, including compression, filtering, and feature extraction. Efficiency of a representation refers to the ability to capture significant information of an object of interest in a small description. For practical applications, this representation has to be realized by structured transforms and fast algorithms. Recently, it has become evident that commonly used separable transforms (such as wavelets) are not necessarily best suited for images. Thus, there is a strong motivation to search for more powerful schemes that can capture the intrinsic geometrical structure of pictorial information. This thesis focuses on the development of new "true" two-dimensional representations for images. The emphasis is on the discrete framework that can lead to algorithmic implementations. The first method constructs multiresolution, local and directional image expansions by using non-separable filter banks. This discrete transform is developed in connection with the continuous-space curvelet construction in harmonic analysis. As a result, the proposed transform provides an efficient representation for two-dimensional piecewise smooth signals that resemble images. The link between the developed filter banks and the continuous-space constructions is set up in a newly defined directional multiresolution analysis. The second method constructs a new family of block directional and orthonormal transforms based on the ridgelet idea, and thus offers an efficient representation for images that are smooth away from straight edges. Finally, directional multiresolution image representations are employed together with statistical modeling, leading to powerful texture models and successful image retrieval systems
Modeling spatial and temporal textures
Thesis (Ph. D.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1997.Includes bibliographical references (leaves 155-161).by Fang Liu.Ph.D
- …