Endmember learning with k-means through SCD model in hyperspectral scene reconstructions

This paper proposes a simple yet effective method for improving the efficiency of sparse coding dictionary learning (DL) with an implication of enhancing the ultimate usefulness of compressive sensing (CS) technology for practical applications, such as in hyperspectral imaging (HSI) scene reconstruction. CS is the technique which allows sparse signals to be decomposed into a sparse representation “a” of a dictionary Du" role="presentation" style="max-height: none; display: inline; line-height: normal; word-spacing: normal; overflow-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; position: relative;">Du . The goodness of the learnt dictionary has direct impacts on the quality of the end results, e.g., in the HSI scene reconstructions. This paper proposes the construction of a concise and comprehensive dictionary by using the cluster centres of the input dataset, and then a greedy approach is adopted to learn all elements within this dictionary. The proposed method consists of an unsupervised clustering algorithm (K-Means), and it is then coupled with an advanced sparse coding dictionary (SCD) method such as the basis pursuit algorithm (orthogonal matching pursuit, OMP) for the dictionary learning. The effectiveness of the proposed K-Means Sparse Coding Dictionary (KMSCD) is illustrated through the reconstructions of several publicly available HSI scenes. The results have shown that the proposed KMSCD achieves ~40% greater accuracy, 5 times faster convergence and is twice as robust as that of the classic Spare Coding Dictionary (C-SCD) method that adopts random sampling of data for the dictionary learning. Over the five data sets that have been employed in this study, it is seen that the proposed KMSCD is capable of reconstructing these scenes with mean accuracies of approximately 20–500% better than all competing algorithms adopted in this work. Furthermore, the reconstruction efficiency of trace materials in the scene has been assessed: it is shown that the KMSCD is capable of recovering ~12% better than that of the C-SCD. These results suggest that the proposed DL using a simple clustering method for the construction of the dictionary has been shown to enhance the scene reconstruction substantially. When the proposed KMSCD is incorporated with the Fast non-negative orthogonal matching pursuit (FNNOMP) to constrain the maximum number of materials to coexist in a pixel to four, experiments have shown that it achieves approximately ten times better than that constrained by using the widely employed TMM algorithm. This may suggest that the proposed DL method using KMSCD and together with the FNNOMP will be more suitable to be the material allocation module of HSI scene simulators like the CameoSim packag

a critical examination and new developments

2012-2013Remote sensing consists in measuring some characteristics of an object from a distance. A key example of remote sensing is the Earth observation from sensors mounted on satellites that is a crucial aspect of space programs. The first satellite used for Earth observation was Explorer VII. It has been followed by thousands of satellites, many of which are still working. Due to the availability of a large number of different sensors and the subsequent huge amount of data collected, the idea of obtaining improved products by means of fusion algorithms is becoming more intriguing. Data fusion is often exploited for indicating the process of integrating multiple data and knowledge related to the same real-world scene into a consistent, accurate, and useful representation. This term is very generic and it includes different levels of fusion. This dissertation is focused on the low level data fusion, which consists in combining several sources of raw data. In this field, one of the most relevant scientific application is surely the Pansharpening. Pansharpening refers to the fusion of a panchromatic image (a single band that covers the visible and near infrared spectrum) and a multispectral/hyperspectral image (tens/hundreds bands) acquired on the same area. [edited by author]XII ciclo n.s

On the decomposition of Mars hyperspectral data by ICA and Bayesian positive source separation

International audienceThe surface of Mars is currently being imaged with an unprecedented combination of spectral and spatial resolution. This high resolution, and its spectral range, gives the ability to pinpoint chemical species on the surface and the atmosphere of Mars more accurately than before. The subject of this paper is to present a method to extract informations on these chemicals from hyperspectral images. A first approach, based on independent component analysis (ICA) [P. Comon, Independent component analysis, a new concept? Signal Process. 36 (3) (1994) 287-314], is able to extract artifacts and locations of CO2 and H2O ices. However, the main independence assumption and some basic properties (like the positivity of images and spectra) being unverified, the reliability of all the independent components (ICs) is weak. For improving the component extraction and consequently the endmember classification, a combination of spatial ICA with spectral Bayesian positive source separation (BPSS) [S. Moussaoui, D. Brie, A. Mohammad-Djafari, C. Carteret, Separation of non-negative mixture of non-negative sources using a Bayesian approach and MCMC sampling, IEEE Trans. Signal Process. 54 (11) (2006) 4133-4145] is proposed. To reduce the computational burden, the basic idea is to use spatial ICA yielding a rough classification of pixels, which allows selection of small, but relevant, number of pixels. Then, BPSS is applied for the estimation of the source spectra using the spectral mixtures provided by this reduced set of pixels. Finally, the abundances of the components are assessed on the whole pixels of the images. Results of this approach are shown and evaluated by comparison with available reference spectra

Interpretable Hyperspectral AI: When Non-Convex Modeling meets Hyperspectral Remote Sensing

Hyperspectral imaging, also known as image spectrometry, is a landmark technique in geoscience and remote sensing (RS). In the past decade, enormous efforts have been made to process and analyze these hyperspectral (HS) products mainly by means of seasoned experts. However, with the ever-growing volume of data, the bulk of costs in manpower and material resources poses new challenges on reducing the burden of manual labor and improving efficiency. For this reason, it is, therefore, urgent to develop more intelligent and automatic approaches for various HS RS applications. Machine learning (ML) tools with convex optimization have successfully undertaken the tasks of numerous artificial intelligence (AI)-related applications. However, their ability in handling complex practical problems remains limited, particularly for HS data, due to the effects of various spectral variabilities in the process of HS imaging and the complexity and redundancy of higher dimensional HS signals. Compared to the convex models, non-convex modeling, which is capable of characterizing more complex real scenes and providing the model interpretability technically and theoretically, has been proven to be a feasible solution to reduce the gap between challenging HS vision tasks and currently advanced intelligent data processing models

Adaptive distance-based band hierarchy (ADBH) for effective hyperspectral band selection.

Band selection has become a significant issue for the efficiency of the hyperspectral image (HSI) processing. Although many unsupervised band selection (UBS) approaches have been developed in the last decades, a flexible and robust method is still lacking. The lack of proper understanding of the HSI data structure has resulted in the inconsistency in the outcome of UBS. Besides, most of the UBS methods are either relying on complicated measurements or rather noise sensitive, which hinder the efficiency of the determined band subset. In this article, an adaptive distance-based band hierarchy (ADBH) clustering framework is proposed for UBS in HSI, which can help to avoid the noisy bands while reflecting the hierarchical data structure of HSI. With a tree hierarchy-based framework, we can acquire any number of band subset. By introducing a novel adaptive distance into the hierarchy, the similarity between bands and band groups can be computed straightforward while reducing the effect of noisy bands. Experiments on four datasets acquired from two HSI systems have fully validated the superiority of the proposed framework

Robust hyperspectral image reconstruction for scene simulation applications

This thesis presents the development of a spectral reconstruction method for multispectral (MSI) and hyperspectral (HSI) applications through an enhanced dictionary learning and spectral unmixing methodologies. Earth observation/surveillance is largely undertaken by MSI sensing such as that given by the Landsat, WorldView, Sentinel etc, however, the practical usefulness of the MSI data set is very limited. This is mainly because of the very limited number of wave bands that can be provided by the MSI imagery. One means to remedy this major shortcoming is to extend the MSI into HSI without the need of involving expensive hardware investment. Specifically, spectral reconstruction has been one of the most critical elements in applications such as Hyperspectral scene simulation. Hyperspectral scene simulation has been an important technique particularly for defence applications. Scene simulation creates a virtual scene such that modelling of the materials in the scene can be tailored freely to allow certain parameters of the model to be studied. In the defence sector this is the most cost-effective technique to allow the vulnerability of the soldiers/vehicles to be evaluated before they are deployed to a foreign ground. The simulation of a hyperspectral scene requires the details of materials in the scene, which is normally not available. Current state-of-the-art technology is trying to make use of the MSI satellite data, and to transform it into HSI for the hyperspectral scene simulation. One way to achieve this is through a reconstruction algorithm, commonly known as spectral reconstruction, which turns the MSI into HSI using an optimisation approach. The methodology that has been adopted in this thesis is the development of a robust dictionary learning to estimate the endmember (EM) robustly. Once the EM is found the abundance of materials in the scene can be subsequently estimated through a linear unmixing approach. Conventional approaches to the material allocation of most Hyperspectral scene simulator has been using the Texture Material Mapper (TMM) algorithm, which allocates materials from a spectral library (a collection of pre-compiled endmember iii iv materials) database according to the minimum spectral Euclidean distance difference to a candidate pixel of the scene. This approach has been shown (in this work) to be highly inaccurate with large scene reconstruction error. This research attempts to use a dictionary learning technique for material allocation, solving it as an optimisation problem with the objective of: (i) to reconstruct the scene as closely as possible to the ground truth with a fraction of error as that given by the TMM method, and (ii) to learn materials which are trace (2-3 times the number of species (i.e. intrinsic dimension) in the scene) cluster to ensure all material species in the scene is included for the scene reconstruction. Furthermore, two approaches complementing the goals of the learned dictionary through a rapid orthogonal matching pursuit (r-OMP) which enhances the performance of the orthogonal matching pursuit algorithm; and secondly a semi-blind approximation of the irradiance of all pixels in the scene including those in the shaded regions, have been proposed in this work. The main result of this research is the demonstration of the effectiveness of the proposed algorithms using real data set. The SCD-SOMP has been shown capable to learn both the background and trace materials even for a dictionary with small number of atoms (≈10). Also, the KMSCD method is found to be the more versatile with overcomplete (non-orthogonal) dictionary capable to learn trace materials with high scene reconstruction accuracy (2x of accuracy enhancement over that simulated using the TMM method. Although this work has achieved an incremental improvement in spectral reconstruction, however, the need of dictionary training using hyperspectral data set in this thesis has been identified as one limitation which is needed to be removed for the future direction of research

Image Fusion in Remote Sensing and Quality Evaluation of Fused Images

In remote sensing, acquired optical images of high spectral resolution have usually a lower spatial resolution than images of lower spectral resolution. This is due to physical, cost and complexity constraints. To make the most of the available imagery, many image fusion techniques have been developed to address this problem. Image fusion is an ill-posed inverse problem where an image of low spatial resolution and high spectral resolution is enhanced in spatial-resolution by using an auxiliary image of high spatial resolution and low spectral resolution. It is assumed that both images display the same scene and are properly co-registered. Thus, the problem is essentially to transfer details from the higher spatial resolution auxiliary image to the upscaled lower resolution image in a manner that minimizes the spatial and spectral distortion of the fused image. The most common image fusion problem is pansharpening, where a multispectral (MS) image is enhanced using wide-band panchromatic (PAN) image. A similar problem is the enhancement of a hyperspectral (HS) image by either a PAN image or an MS image. As there is no reference image available, the reliable quantitative evaluation of the quality of the fused image is a difficult problem. This thesis addresses the image fusion problem in three different ways and also addresses the problem of quantitative quality evaluation.Í fjarkönnun hafa myndir með háa rófsupplausn lægri rúmupplausn en myndir með lægri rófsupplausn vegna eðlisfræðilegra og kostnaðarlegra takmarkana. Til að auka upplýsingamagn slíkra mynda hafa verið þróaðar fjölmargar sambræðsluaðferðir á síðustu tveimur áratugum. Myndsambræðsla er illa framsett andhverft vandmál (e. inverse problem) þar sem rúmupplausn myndar af hárri rófsupplausn er aukin með því að nota upplýsingar frá mynd af hárri rúmupplausn og lægri rófsupplausn. Það er gert ráð fyrir að báðar myndir sýni nákvæmlega sama landsvæði. Þannig er vandamálið í eðli sínu að flytja fíngerða eiginleika myndar af hærri rúmupplausn yfir á mynd af lægri rúmupplausn sem hefur verið brúuð upp í stærð hinnar myndarinnar, án þess að skerða gæði rófsupplýsinga upphaflegu myndarinnar. Algengasta myndbræðsluvandamálið í fjarkönnun er svokölluð panskerpun (e. pansharpening) þar sem fjölrásamynd (e. multispectral image) er endurbætt í rúmi með svokallaðri víðbandsmynd (e. panchromatic image) sem hefur aðeins eina rás af hárri upplausn. Annað svipað vandamál er sambræðsla háfjölrásamyndar (e. hyperspectral image) og annaðhvort fjölrásamyndar eða víðbandsmyndar. Þar sem myndsambræðsla er andhverft vandmál er engin háupplausnar samanburðarmynd tiltæk, sem gerir mat á gæðum sambræddu myndarinnar að erfiðu vandamáli. Í þessari ritgerð eru kynntar þrjár aðferðir sem taka á myndsambræðlsu og einnig er fjallað um mat á gæðum sambræddra mynda, þá sérstaklega panskerptra mynda

Hyperspectral unmixing: a theoretical aspect and applications to CRISM data processing

Hyperspectral imaging has been deployed in earth and planetary remote sensing, and has contributed the development of new methods for monitoring the earth environment and new discoveries in planetary science. It has given scientists and engineers a new way to observe the surface of earth and planetary bodies by measuring the spectroscopic spectrum at a pixel scale. Hyperspectal images require complex processing before practical use. One of the important goals of hyperspectral imaging is to obtain the images of reflectance spectrum. A raw image obtained by hyperspectral remote sensing usually undergoes conversion to a physical quantity representing the intensity of light energy, called radiance. In order to obtain the reflectance spectrum of surface, the contribution of atmosphere needs to be addressed and then divided by a spectrum of ``white reference.\u27\u27 Furthermore, the obtained reflectance spectra of image pixels are likely to be the mixtures of multiple species due to limited spatial resolution from orbits around planets. Hyperspectral unmixing is an attempt to unmix those pixels - to identify substantial components and estimate their fractional abundances. Hyperspectral unmixing has been widely explored in the literature, but there are still many aspects yet to be studied. The majority of research focuses on the development of methods to retrieve correct substantial components and accurate fractional abundances. Their theoretical aspects are rarely investigated. Chapter 2 will pursue a theoretical aspect of sparse unmixing, one of the hyperspectral unmixing problems and derive its theoretical conditions that guarantee the correct identification of substantial components. Hyperspectral unmixing can also be used for other stages of hyperspectral data processing. Chapter 3 explores the application of hyperspectral unmixing to the processing of hyperspectral image acquired by the Compact Reconnaissance Imaging Spectrometer for Mars (CRISM) onboard the Mars Reconnaissance Orbiter (MRO). In particular, new atmospheric correction and de-noising methods for the CRISM data that use a hyperspectral unmixing to model surface spectra, are introduced. The new methods remove most of the problematic systematic artifacts present in CRISM images and significantly improve signal quality. Chapter 4 investigates how hyperspectral images acquired from orbits can be combined with ground exploration. In the recent rush of the launch of many Martian ground rover missions, it is important to effectively integrate knowledge obtained by hyperspectral remote sensing from orbits into ground exploration for facilitating Martian exploration. In specific, this dissertation solves the problem of matching hyperspectral image pixels obtained by the CRISM with ground mega-pixel images acquired by the Mast Camera (Mastcam) installed on the Curiosity rover on Mars. A new systematic methodology to map the CRISM and Mastcam images onto high resolution surface topography is developed

Nonconvex Optimization Algorithms for Structured Matrix Estimation in Large-Scale Data Applications

Το πρόβλημα της εκτίμησης δομημένου πίνακα ανήκει στην κατηγορία των προβλημάτων εύρεσης αναπαραστάσεων χαμηλής διάστασης (low-dimensional embeddings) σε δεδομένα υψηλής διάστασης. Στις μέρες μας συναντάται σε μια πληθώρα εφαρμογών που σχετίζονται με τις ερευνητικές περιοχές της επεξεργασίας σήματος και της μηχανικής μάθησης. Στην παρούσα διατριβή προτείνονται νέοι μαθηματικοί φορμαλισμοί σε τρία διαφορετικά προβλήματα εκτίμησης δομημένων πινάκων από δεδομένα μεγάλης κλίμακας. Πιο συγκεκριμένα, μελετώνται τα ερευνητικά προβλήματα α) της εκτίμησης πίνακα που είναι ταυτόχρονα αραιός, χαμηλού βαθμού και μη-αρνητικός, β) της παραγοντοποίησης πίνακα χαμηλού βαθμού, και γ) της ακολουθιακής (online) εκτίμησης πίνακα υποχώρου (subspace matrix) χαμηλού βαθμού από ελλιπή δεδομένα. Για όλα τα προβλήματα αυτά προτείνονται καινoτόμοι και αποδοτικοί αλγόριθμοι βελτιστοποίησης (optimization algorithms). Βασική υπόθεση που υιοθετείται σε κάθε περίπτωση είναι πως τα δεδομένα έχουν παραχθεί με βάση ένα γραμμικό μοντέλο. Το σύνολο των προσεγγίσεων που ακολουθούνται χαρακτηρίζονται από μη-κυρτότητα. Όπως γίνεται φανερό στην παρούσα διατριβή, η ιδιότητα αυτή, παρά τις δυσκολίες που εισάγει στην θεωρητική τεκμηρίωση των προτεινόμενων μεθόδων (σε αντίθεση με τις κυρτές προσεγγίσεις στις οποίες η θεωρητική ανάλυση είναι σχετικά ευκολότερη), οδηγεί σε σημαντικά οφέλη όσον αφορά την απόδοσή τους σε πλήθος πραγματικών εφαρμογών. Για την εκτίμηση πίνακα που είναι ταυτόχρονα αραιός, χαμηλού βαθμού και μη-αρνητικός, προτείνονται στην παρούσα διατριβή τρεις νέοι αλγόριθμοι, από τους οποίους οι δύο πρώτοι ελαχιστοποιούν μια κοινή συνάρτηση κόστους και ο τρίτος μια ελαφρώς διαφορετική συνάρτηση κόστους. Κοινό χαρακτηριστικό και των δύο αυτών συναρτήσεων είναι ότι κατά βάση αποτελούνται από έναν όρο προσαρμογής στα δεδομένα και δύο όρους κανονικοποίησης, οι οποίοι χρησιμοποιούνται για την επιβολή αραιότητας και χαμηλού βαθμού, αντίστοιχα. Στην πρώτη περίπτωση αυτό επιτυγχάνεται με την αξιοποίηση του αθροίσματος της επανασταθμισμένης l1 νόρμας (reweighted l1 norm) και της επανασταθμισμένης πυρηνικής νόρμας (reweighted nuclear norm), οι οποίες ευθύνονται για το μη- κυρτό χαρακτήρα της προκύπτουσας συνάρτησης κόστους. Από τους δύο προτεινόμενους αλγορίθμους που ελαχιστοποιούν τη συνάρτηση αυτή, ο ένας ακολουθεί τη μέθοδο καθόδου σταδιακής εγγύτητας και ο άλλος βασίζεται στην πιο απαιτητική υπολογιστικά μέθοδο ADMM. Η δεύτερη συνάρτηση κόστους διαφοροποιείται σε σχέση με την πρώτη καθώς χρησιμοποιεί μια προσέγγιση παραγοντοποίησης για τη μοντελοποίηση του χαμηλού βαθμού του δομημένου πίνακα. Επιπλέον, λόγω της μη εκ των προτέρων γνώσης του πραγματικού βαθμού, ενσωματώνει έναν όρο επιβολής χαμηλού βαθμού, μέσω της μη- κυρτής έκφρασης που έχει προταθεί ως ένα άνω αυστηρό φράγμα της (κυρτής) πυρηνικής νόρμας (σ.σ. στο εξής θα αναφέρεται ως εναλλακτική μορφή της πυρηνικής νόρμας). Και στην περίπτωση αυτή, το πρόβλημα που προκύπτει είναι μη-κυρτό λόγω του φορμαλισμού του μέσω της παραγοντοποίησης πίνακα, ενώ η βελτιστοποίηση πραγματοποιείται εφαρμόζοντας μια υπολογιστικά αποδοτική μέθοδο καθόδου συνιστωσών ανά μπλοκ (block coordinate descent). Tο σύνολο των προτεινόμενων σχημάτων χρησιμοποιείται για τη μοντελοποίηση, με καινοτόμο τρόπο, του προβλήματος φασματικού διαχωρισμού υπερφασματικών εικόνων (ΥΦΕ). Όπως εξηγείται αναλυτικά, τόσο η αραιότητα όσο και ο χαμηλός βαθμός παρέχουν πολύτιμες ερμηνείες ορισμένων φυσικών χαρακτηριστικών των ΥΦΕ, όπως π.χ. η χωρική συσχέτιση. Πιο συγκεκριμένα, η αραιότητα και ο χαμηλός βαθμός μπορούν να υιοθετηθούν ως δομές στον πίνακα αφθονίας (abundance matrix - ο πίνακας που περιέχει τα ποσοστά παρουσίας των υλικών στην περιοχή που απεικονίζει κάθε εικονοστοιχείο). Τα σημαντικά πλεονεκτήματα που προσφέρουν οι προτεινόμενες τεχνικές, σε σχέση με ανταγωνιστικούς αλγορίθμους, αναδεικνύονται σε ένα πλήθος διαφορετικών πειραμάτων που πραγματοποιούνται τόσο σε συνθετικά όσο και σε αληθινά υπερφασματικά δεδομένα. Στο πλαίσιο της παραγοντοποίησης πίνακα χαμηλού βαθμού (low-rank matrix factorization) περιγράφονται στη διατριβή τέσσερις νέοι αλγόριθμοι, ο καθένας εκ των οποίων έχει σχεδιαστεί για μια διαφορετική έκφανση του συγκεκριμένου προβλήματος. Όλα τα προτεινόμενα σχήματα έχουν ένα κοινό χαρακτηριστικό: επιβάλλουν χαμηλό βαθμό στους πίνακες-παράγοντες καθώς και στο γινόμενό τους με την εισαγωγή ενός νέου όρου κανονικοποίησης. Ο όρος αυτός προκύπτει ως μια γενίκευση της εναλλακτικής έκφρασης της πυρηνικής νόρμας με τη μετατροπή της σε σταθμισμένη μορφή. Αξίζει να επισημανθεί πως με κατάλληλη επιλογή των πινάκων στάθμισης καταλήγουμε σε μια ειδική έκφραση της συγκεκριμένης νόρμας η οποία ανάγει την διαδικασία επιβολής χαμηλού βαθμού σε αυτή της από κοινού επιβολής αραιότητας στις στήλες των δύο πινάκων. Όπως αναδεικνύεται αναλυτικά, η ιδιότητα αυτή είναι πολύ χρήσιμη ιδιαιτέρως σε εφαρμογές διαχείρισης δεδομένων μεγάλης κλίμακας. Στα πλαίσια αυτά μελετώνται τρία πολύ σημαντικά προβλήματα στο πεδίο της μηχανικής μάθησης και συγκεκριμένα αυτά της αποθορυβοποίησης σήματος (denoising), πλήρωσης πίνακα (matrix completion) και παραγοντοποίησης μη-αρνητικού πίνακα (nonnegative matrix factorization). Χρησιμοποιώντας τη μέθοδο ελαχιστοποίησης άνω φραγμάτων συναρτήσεων διαδοχικών μπλοκ (block successive upper bound minimization) αναπτύσσονται τρεις νέοι επαναληπτικά σταθμισμένοι αλγόριθμοι τύπου Newton, οι οποίοι σχεδιάζονται κατάλληλα, λαμβάνοντας υπόψη τα ιδιαίτερα χαρακτηριστικά του εκάστοτε προβλήματος. Τέλος, παρουσιάζεται αλγόριθμος παραγοντοποίησης πίνακα ο οποίος έχει σχεδιαστεί πάνω στην προαναφερθείσα ιδέα επιβολής χαμηλού βαθμού, υποθέτοντας παράλληλα αραιότητα στον ένα πίνακα-παράγοντα. Η επαλήθευση της αποδοτικότητας όλων των αλγορίθμων που εισάγονται γίνεται με την εφαρμογή τους σε εκτεταμένα συνθετικά πειράματα, όπως επίσης και σε εφαρμογές πραγματικών δεδομένων μεγάλης κλίμακας π.χ. αποθορυβοποίηση ΥΦΕ, πλήρωση πινάκων από συστήματα συστάσεων (recommender systems) ταινιών, διαχωρισμός μουσικού σήματος και τέλος μη-επιβλεπόμενος φασματικός διαχωρισμός. Το τελευταίο πρόβλημα το οποίο διαπραγματεύεται η παρούσα διατριβή είναι αυτό της ακολουθιακής εκμάθησης υποχώρου χαμηλού βαθμού και της πλήρωσης πίνακα. Το πρόβλημα αυτό εδράζεται σε ένα διαφορετικό πλαίσιο μάθησης, την επονομαζόμενη ακολουθιακή μάθηση, η οποία αποτελεί μια πολύτιμη προσέγγιση σε εφαρμογές δεδομένων μεγάλης κλίμακας, αλλά και σε εφαρμογές που λαμβάνουν χώρα σε χρονικά μεταβαλλόμενα περιβάλλοντα. Στην παρούσα διατριβή προτείνονται δύο διαφορετικοί αλγόριθμοι, ένας μπεϋζιανός και ένας ντετερμινιστικός. Ο πρώτος αλγόριθμος προκύπτει από την εφαρμογή μιας καινοτόμου ακολουθιακής μεθόδου συμπερασμού βασισμένου σε μεταβολές. Αυτή η μέθοδος χρησιμοποιείται για την πραγματοποίηση προσεγγιστικού συμπερασμού στο προτεινόμενο ιεραρχικό μπεϋζιανό μοντέλο. Αξίζει να σημειωθεί πως το μοντέλο αυτό έχει σχεδιαστεί με κατάλληλο τρόπο έτσι ώστε να ενσωματώνει, σε πιθανοτικό πλαίσιο, την ίδια ιδέα επιβολής χαμηλού βαθμού που προτείνεται για το πρόβλημα παραγοντοποίησης πίνακα χαμηλού βαθμού, δηλαδή επιβάλλοντας από-κοινού αραιότητα στους πίνακες-παράγοντες. Ωστόσο, ακολουθώντας την πιθανοτική προσέγγιση, αυτό πραγματοποιείται επιβάλλοντας πολύ-επίπεδες a priori κατανομές Laplace στις στήλες τους. Ο αλγόριθμος που προκύπτει είναι πλήρως αυτοματοποιημένος, μιας και δεν απαιτεί τη ρύθμιση κάποιας παραμέτρου κανονικοποίησης. Ο δεύτερος αλγόριθμος προκύπτει από την ελαχιστοποίηση μιας κατάλληλα διαμορφωμένης συνάρτησης κόστους. Και στην περίπτωση αυτή, χρησιμοποιείται η προαναφερθείσα ιδέα επιβολής χαμηλού βαθμού (κατάλληλα τροποποιημένη έτσι ώστε να μπορεί να εφαρμοστεί στο ακολουθιακό πλαίσιο μάθησης). Ενδιαφέρον παρουσιάζει το γεγονός πως ο τελευταίος αλγόριθμος μπορεί να θεωρηθεί ως μια ντετερμινιστική εκδοχή του προαναφερθέντος πιθανοτικού αλγορίθμου. Τέλος, σημαντικό χαρακτηριστικό και των δύο αλγορίθμων είναι ότι δεν είναι απαραίτητη η εκ των προτέρων γνώση του βαθμού του πίνακα υποχώρου. Τα πλεονεκτήματα των προτεινόμενων προσεγγίσεων παρουσιάζονται σε ένα μεγάλο εύρος πειραμάτων που πραγματοποιήθηκαν σε συνθετικά δεδομένα, στο πρόβλημα της ακολουθιακής πλήρωσης ΥΦΕ και στην εκμάθηση ιδιο-προσώπων κάνοντας χρήση πραγματικών δεδομένων.Structured matrix estimation belongs to the family of learning tasks whose main goal is to reveal low-dimensional embeddings of high-dimensional data. Nowadays, this task appears in various forms in a plethora of signal processing and machine learning applications. In the present thesis, novel mathematical formulations for three different instances of structured matrix estimation are proposed. Concretely, the problems of a) simultaneously sparse, low-rank and nonnegative matrix estimation, b) low-rank matrix factorization and c) online low-rank subspace learning and matrix completion, are addressed and analyzed. In all cases, it is assumed that data are generated by a linear process, i.e., we deal with linear measurements. A suite of novel and efficient {\it optimization algorithms} amenable to handling {\it large-scale data} are presented. A key common feature of all the introduced schemes is {\it nonconvexity}. It should be noted that albeit nonconvexity complicates the derivation of theoretical guarantees (contrary to convex relevant approaches, which - in most cases - can be theoretically analyzed relatively easily), significant gains in terms of the estimation performance of the emerging algorithms have been recently witnessed in several real practical situations. Let us first focus on simultaneously sparse, low-rank and nonnegative matrix estimation from linear measurements. In the thesis this problem is resolved by three different optimization algorithms, which address two different and novel formulations of the relevant task. All the proposed schemes are suitably devised for minimizing a cost function consisting of a least-squares data fitting term and two regularization terms. The latter are utilized for promoting sparsity and low-rankness. The novelty of the first formulation lies in the use, for the first time in the literature, of the sum of the reweighted $\ell_1$ and the reweighted nuclear norms. The merits of reweighted $\ell_1$ and nuclear norms have been exposed in numerous sparse and low-rank matrix recovery problems. As is known, albeit these two norms induce nonconvexity in the resulting optimization problems, they provide a better approximation of the $\ell_0$ norm and the rank function, respectively, as compared to relevant convex regularizers. Herein, we aspire to benefit from the use of the combination of these two norms. The first algorithm is an incremental proximal minimization scheme, while the second one is an ADMM solver. The third algorithm's main goal is to further reduce the computational complexity. Towards this end, it deviates from the other two in the use of a matrix factorization based approach for modelling low-rankness. Since the rank of the sought matrix is generally unknown, a low-rank imposing term, i.e., the variational form of the nuclear norm, which is a function of the matrix factors, is utilized. In this case, the optimization process takes place via a block coordinate descent type scheme. The proposed formulations are utilized for modelling in a pioneering way a very important problem in hyperspectral image processing, that of hyperspectral image unmixing. It is shown that both sparsity and low-rank offer meaningful interpretations of inherent natural characteristics of hyperspectral images. More specifically, both sparsity and low-rankness are reasonable hypotheses that can be made for the so-called {\it abundance} matrix, i.e., the nonnegative matrix containing the fractions of presence of the different materials, called {\it endmembers}, at the region depicted by each pixel. The merits of the proposed algorithms over other state-of-the-art hyperspectral unmixing algorithms are corroborated in a wealth of simulated and real hyperspectral imaging data experiments. In the framework of low-rank matrix factorization (LRMF) four novel optimization algorithms are presented, each modelling a different instance of it. All the proposed schemes share a common thread: they impose low-rank on both matrix factors and the sought matrix by a newly introduced regularization term. This term can be considered as a generalized weighted version of the variational form of the nuclear norm. Notably, by appropriately selecting the weight matrix, low-rank enforcement amounts to imposing joint column sparsity on both matrix factors. This property is actually proven to be quite important in applications dealing with large-scale data, since it leads to a significant decrease of the induced computational complexity. Along these lines, three well-known machine learning tasks, namely, denoising, matrix completion and low-rank nonnegative matrix factorization (NMF), are redefined according to the new low-rank regularization approach. Then, following the block successive upper bound minimization framework, alternating iteratively reweighted least-squares, Newton-type algorithms are devised accounting for the particular characteristics of the problem that each time is addressed. Lastly, an additional low-rank and sparse NMF algorithm is proposed, which hinges upon the same low-rank promoting idea mentioned above, while also accounting for sparsity on one of the matrix factors. All the derived algorithms are tested on extensive simulated data experiments and real large-scale data applications such as hyperspectral image denoising, matrix completion for recommender systems, music signal decomposition and unsupervised hyperspectral image unmixing with unknown number of endmembers. The last problem that this thesis touches upon is online low-rank subspace learning and matrix completion. This task follows a different learning model, i.e., online learning, which offers a valuable processing framework when one deals with large-scale streaming data possibly under time-varying conditions. In the thesis, two different online algorithms are put forth. The first one stems from a newly developed online variational Bayes scheme. This is applied for performing approximate inference based on a carefully designed novel multi-hierarchical Bayesian model. Notably, the adopted model encompasses similar low-rank promoting ideas to those mentioned for LRMF. That is, low-rank is imposed via promoting jointly column sparsity on the columns of the matrix factors. However, following the Bayesian rationale, this now takes place by assigning Laplace-type marginal priors on the matrix factors. Going one step further, additional sparsity is independently modelled on the subspace matrix thus imposing multiple structures on the same matrix. The resulting algorithm is fully automated, i.e., it does not demand fine-tuning of any parameters. The second algorithm follows a cost function minimization based strategy. Again, the same low-rank promoting idea introduced for LRMF is incorporated in this problem via the use of a - modified to the online processing scenario - low-rank regularization term. Interestingly, the resulting optimization scheme can be considered as the deterministic analogue of the Bayesian one. Both the proposed algorithms present a favorable feature, i.e., they are competent to learn subspaces without requiring the a priori knowledge of their true rank. Their effectiveness is showcased in extensive simulated data experiments and in online hyperspectral image completion and eigenface learning using real data