Estimating the Laplacian matrix of Gaussian mixtures for signal processing on graphs

[EN] Recent works in signal processing on graphs have been driven to estimate the precision matrix and to use it as the graph Laplacian matrix. The normalized elements of the precision matrix are the partial correlation coefficients which measure the pairwise conditional linear dependencies of the graph. However, the non-linear dependencies inherent in any non-Gaussian model cannot be captured. We propose in this paper a generalized partial correlation coefficient which is derived by assuming an underlying multivariate Gaussian Mixture Model of the observations. Exact and approximate methods are proposed to estimate the generalized partial correlation coefficients from estimates of the Gaussian Mixture Model parameters. Thus it may find application in any non-Gaussian scenario where the Laplacian matrix is to be learned from training signals. (C) 2018 Elsevier B.V. All rights reserved.This work was supported by Spanish Administration (Ministerio de Economia y Competitividad) and European Union (FEDER) under grant TEC2014-58438-R, and Generalitat Valenciana under grant PROMETEO II/2014/032.Belda, J.; Vergara Domínguez, L.; Salazar Afanador, A.; Safont Armero, G. (2018). Estimating the Laplacian matrix of Gaussian mixtures for signal processing on graphs. Signal Processing. 148:241-249. https://doi.org/10.1016/j.sigpro.2018.02.017S24124914

A New Surrogating Algorithm by the Complex Graph Fourier Transform (CGFT)

[EN] The essential step of surrogating algorithms is phase randomizing the Fourier transform while preserving the original spectrum amplitude before computing the inverse Fourier transform. In this paper, we propose a new method which considers the graph Fourier transform. In this manner, much more flexibility is gained to define properties of the original graph signal which are to be preserved in the surrogates. The complex case is considered to allow unconstrained phase randomization in the transformed domain, hence we define a Hermitian Laplacian matrix that models the graph topology, whose eigenvectors form the basis of a complex graph Fourier transform. We have shown that the Hermitian Laplacian matrix may have negative eigenvalues. We also show in the paper that preserving the graph spectrum amplitude implies several invariances that can be controlled by the selected Hermitian Laplacian matrix. The interest of surrogating graph signals has been illustrated in the context of scarcity of instances in classifier training.This research was funded by the Spanish Administration and the European Union under grant TEC2017-84743-P.Belda, J.; Vergara Domínguez, L.; Safont Armero, G.; Salazar Afanador, A.; Parcheta, Z. (2019). A New Surrogating Algorithm by the Complex Graph Fourier Transform (CGFT). Entropy. 21(8):1-18. https://doi.org/10.3390/e21080759S118218Schreiber, T., & Schmitz, A. (2000). Surrogate time series. Physica D: Nonlinear Phenomena, 142(3-4), 346-382. doi:10.1016/s0167-2789(00)00043-9Miralles, R., Vergara, L., Salazar, A., & Igual, J. (2008). Blind detection of nonlinearities in multiple-echo ultrasonic signals. IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control, 55(3), 637-647. doi:10.1109/tuffc.2008.688Mandic, D. ., Chen, M., Gautama, T., Van Hulle, M. ., & Constantinides, A. (2008). On the characterization of the deterministic/stochastic and linear/nonlinear nature of time series. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 464(2093), 1141-1160. doi:10.1098/rspa.2007.0154Rios, R. A., Small, M., & de Mello, R. F. (2015). Testing for Linear and Nonlinear Gaussian Processes in Nonstationary Time Series. International Journal of Bifurcation and Chaos, 25(01), 1550013. doi:10.1142/s0218127415500133Borgnat, P., Flandrin, P., Honeine, P., Richard, C., & Xiao, J. (2010). Testing Stationarity With Surrogates: A Time-Frequency Approach. IEEE Transactions on Signal Processing, 58(7), 3459-3470. doi:10.1109/tsp.2010.2043971Shuman, D. I., Narang, S. K., Frossard, P., Ortega, A., & Vandergheynst, P. (2013). The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Magazine, 30(3), 83-98. doi:10.1109/msp.2012.2235192Sandryhaila, A., & Moura, J. M. F. (2013). Discrete Signal Processing on Graphs. IEEE Transactions on Signal Processing, 61(7), 1644-1656. doi:10.1109/tsp.2013.2238935Sandryhaila, A., & Moura, J. M. F. (2014). Big Data Analysis with Signal Processing on Graphs: Representation and processing of massive data sets with irregular structure. IEEE Signal Processing Magazine, 31(5), 80-90. doi:10.1109/msp.2014.2329213Pirondini, E., Vybornova, A., Coscia, M., & Van De Ville, D. (2016). A Spectral Method for Generating Surrogate Graph Signals. IEEE Signal Processing Letters, 23(9), 1275-1278. doi:10.1109/lsp.2016.2594072Sandryhaila, A., & Moura, J. M. F. (2014). Discrete Signal Processing on Graphs: Frequency Analysis. IEEE Transactions on Signal Processing, 62(12), 3042-3054. doi:10.1109/tsp.2014.2321121Shuman, D. I., Ricaud, B., & Vandergheynst, P. (2016). Vertex-frequency analysis on graphs. Applied and Computational Harmonic Analysis, 40(2), 260-291. doi:10.1016/j.acha.2015.02.005Dong, X., Thanou, D., Frossard, P., & Vandergheynst, P. (2016). Learning Laplacian Matrix in Smooth Graph Signal Representations. IEEE Transactions on Signal Processing, 64(23), 6160-6173. doi:10.1109/tsp.2016.2602809Perraudin, N., & Vandergheynst, P. (2017). Stationary Signal Processing on Graphs. IEEE Transactions on Signal Processing, 65(13), 3462-3477. doi:10.1109/tsp.2017.2690388Yu, G., & Qu, H. (2015). Hermitian Laplacian matrix and positive of mixed graphs. Applied Mathematics and Computation, 269, 70-76. doi:10.1016/j.amc.2015.07.045Gilbert, G. T. (1991). Positive Definite Matrices and Sylvester’s Criterion. The American Mathematical Monthly, 98(1), 44-46. doi:10.1080/00029890.1991.11995702Merris, R. (1994). Laplacian matrices of graphs: a survey. Linear Algebra and its Applications, 197-198, 143-176. doi:10.1016/0024-3795(94)90486-3Shapiro, H. (1991). A survey of canonical forms and invariants for unitary similarity. Linear Algebra and its Applications, 147, 101-167. doi:10.1016/0024-3795(91)90232-lFutorny, V., Horn, R. A., & Sergeichuk, V. V. (2017). Specht’s criterion for systems of linear mappings. Linear Algebra and its Applications, 519, 278-295. doi:10.1016/j.laa.2017.01.006Mazumder, R., & Hastie, T. (2012). The graphical lasso: New insights and alternatives. Electronic Journal of Statistics, 6(0), 2125-2149. doi:10.1214/12-ejs740Baba, K., Shibata, R., & Sibuya, M. (2004). PARTIAL CORRELATION AND CONDITIONAL CORRELATION AS MEASURES OF CONDITIONAL INDEPENDENCE. Australian New Zealand Journal of Statistics, 46(4), 657-664. doi:10.1111/j.1467-842x.2004.00360.xChen, X., Xu, M., & Wu, W. B. (2013). Covariance and precision matrix estimation for high-dimensional time series. The Annals of Statistics, 41(6), 2994-3021. doi:10.1214/13-aos1182Theiler, J., Eubank, S., Longtin, A., Galdrikian, B., & Doyne Farmer, J. (1992). Testing for nonlinearity in time series: the method of surrogate data. Physica D: Nonlinear Phenomena, 58(1-4), 77-94. doi:10.1016/0167-2789(92)90102-sSchreiber, T., & Schmitz, A. (1996). Improved Surrogate Data for Nonlinearity Tests. Physical Review Letters, 77(4), 635-638. doi:10.1103/physrevlett.77.635MAMMEN, E., NANDI, S., MAIWALD, T., & TIMMER, J. (2009). EFFECT OF JUMP DISCONTINUITY FOR PHASE-RANDOMIZED SURROGATE DATA TESTING. International Journal of Bifurcation and Chaos, 19(01), 403-408. doi:10.1142/s0218127409022968Lucio, J. H., Valdés, R., & Rodríguez, L. R. (2012). Improvements to surrogate data methods for nonstationary time series. Physical Review E, 85(5). doi:10.1103/physreve.85.056202Schreiber, T. (1998). Constrained Randomization of Time Series Data. Physical Review Letters, 80(10), 2105-2108. doi:10.1103/physrevlett.80.2105Prichard, D., & Theiler, J. (1994). Generating surrogate data for time series with several simultaneously measured variables. Physical Review Letters, 73(7), 951-954. doi:10.1103/physrevlett.73.951Belda, J., Vergara, L., Salazar, A., & Safont, G. (2018). Estimating the Laplacian matrix of Gaussian mixtures for signal processing on graphs. Signal Processing, 148, 241-249. doi:10.1016/j.sigpro.2018.02.017Belda, J., Vergara, L., Safont, G., & Salazar, A. (2018). Computing the Partial Correlation of ICA Models for Non-Gaussian Graph Signal Processing. Entropy, 21(1), 22. doi:10.3390/e21010022Liao, T. W. (2008). Classification of weld flaws with imbalanced class data. Expert Systems with Applications, 35(3), 1041-1052. doi:10.1016/j.eswa.2007.08.044Song, S.-J., & Shin, Y.-K. (2000). Eddy current flaw characterization in tubes by neural networks and finite element modeling. NDT & E International, 33(4), 233-243. doi:10.1016/s0963-8695(99)00046-8Bhattacharyya, S., Jha, S., Tharakunnel, K., & Westland, J. C. (2011). Data mining for credit card fraud: A comparative study. Decision Support Systems, 50(3), 602-613. doi:10.1016/j.dss.2010.08.008Mitra, S., & Acharya, T. (2007). Gesture Recognition: A Survey. IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 37(3), 311-324. doi:10.1109/tsmcc.2007.893280Dardas, N. H., & Georganas, N. D. (2011). Real-Time Hand Gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques. IEEE Transactions on Instrumentation and Measurement, 60(11), 3592-3607. doi:10.1109/tim.2011.2161140Boashash, B. (1992). Estimating and interpreting the instantaneous frequency of a signal. I. Fundamentals. Proceedings of the IEEE, 80(4), 520-538. doi:10.1109/5.135376Horn, A. (1954). Doubly Stochastic Matrices and the Diagonal of a Rotation Matrix. American Journal of Mathematics, 76(3), 620. doi:10.2307/237270

Computing the Partial Correlation of ICA Models for Non-Gaussian Graph Signal Processing

[EN] Conventional partial correlation coefficients (PCC) were extended to the non-Gaussian case, in particular to independent component analysis (ICA) models of the observed multivariate samples. Thus, the usual methods that define the pairwise connections of a graph from the precision matrix were correspondingly extended. The basic concept involved replacing the implicit linear estimation of conventional PCC with a nonlinear estimation (conditional mean) assuming ICA. Thus, it is better eliminated the correlation between a given pair of nodes induced by the rest of nodes, and hence the specific connectivity weights can be better estimated. Some synthetic and real data examples illustrate the approach in a graph signal processing context.This research was funded by Spanish Administration and European Union under grants TEC2014-58438-R and TEC2017-84743-P.Belda, J.; Vergara Domínguez, L.; Safont Armero, G.; Salazar Afanador, A. (2019). Computing the Partial Correlation of ICA Models for Non-Gaussian Graph Signal Processing. Entropy. 21(1):1-16. https://doi.org/10.3390/e21010022S116211Baba, K., Shibata, R., & Sibuya, M. (2004). PARTIAL CORRELATION AND CONDITIONAL CORRELATION AS MEASURES OF CONDITIONAL INDEPENDENCE. Australian New Zealand Journal of Statistics, 46(4), 657-664. doi:10.1111/j.1467-842x.2004.00360.xShuman, D. I., Narang, S. K., Frossard, P., Ortega, A., & Vandergheynst, P. (2013). The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Magazine, 30(3), 83-98. doi:10.1109/msp.2012.2235192Sandryhaila, A., & Moura, J. M. F. (2013). Discrete Signal Processing on Graphs. IEEE Transactions on Signal Processing, 61(7), 1644-1656. doi:10.1109/tsp.2013.2238935Ortega, A., Frossard, P., Kovacevic, J., Moura, J. M. F., & Vandergheynst, P. (2018). Graph Signal Processing: Overview, Challenges, and Applications. Proceedings of the IEEE, 106(5), 808-828. doi:10.1109/jproc.2018.2820126Mazumder, R., & Hastie, T. (2012). The graphical lasso: New insights and alternatives. Electronic Journal of Statistics, 6(0), 2125-2149. doi:10.1214/12-ejs740Chen, X., Xu, M., & Wu, W. B. (2013). Covariance and precision matrix estimation for high-dimensional time series. The Annals of Statistics, 41(6), 2994-3021. doi:10.1214/13-aos1182Friedman, J., Hastie, T., & Tibshirani, R. (2007). Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3), 432-441. doi:10.1093/biostatistics/kxm045Peng, J., Wang, P., Zhou, N., & Zhu, J. (2009). Partial Correlation Estimation by Joint Sparse Regression Models. Journal of the American Statistical Association, 104(486), 735-746. doi:10.1198/jasa.2009.0126Belda, J., Vergara, L., Salazar, A., & Safont, G. (2018). Estimating the Laplacian matrix of Gaussian mixtures for signal processing on graphs. Signal Processing, 148, 241-249. doi:10.1016/j.sigpro.2018.02.017Hyvärinen, A., & Oja, E. (2000). Independent component analysis: algorithms and applications. Neural Networks, 13(4-5), 411-430. doi:10.1016/s0893-6080(00)00026-5Chai, R., Naik, G. R., Nguyen, T. N., Ling, S. H., Tran, Y., Craig, A., & Nguyen, H. T. (2017). Driver Fatigue Classification With Independent Component by Entropy Rate Bound Minimization Analysis in an EEG-Based System. IEEE Journal of Biomedical and Health Informatics, 21(3), 715-724. doi:10.1109/jbhi.2016.2532354Liu, H., Liu, S., Huang, T., Zhang, Z., Hu, Y., & Zhang, T. (2016). Infrared spectrum blind deconvolution algorithm via learned dictionaries and sparse representation. Applied Optics, 55(10), 2813. doi:10.1364/ao.55.002813Naik, G. R., Selvan, S. E., & Nguyen, H. T. (2016). Single-Channel EMG Classification With Ensemble-Empirical-Mode-Decomposition-Based ICA for Diagnosing Neuromuscular Disorders. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 24(7), 734-743. doi:10.1109/tnsre.2015.2454503Guo, Y., Huang, S., Li, Y., & Naik, G. R. (2013). Edge Effect Elimination in Single-Mixture Blind Source Separation. Circuits, Systems, and Signal Processing, 32(5), 2317-2334. doi:10.1007/s00034-013-9556-9Chi, Y. (2016). Guaranteed Blind Sparse Spikes Deconvolution via Lifting and Convex Optimization. IEEE Journal of Selected Topics in Signal Processing, 10(4), 782-794. doi:10.1109/jstsp.2016.2543462Pendharkar, G., Naik, G. R., & Nguyen, H. T. (2014). Using Blind Source Separation on accelerometry data to analyze and distinguish the toe walking gait from normal gait in ITW children. Biomedical Signal Processing and Control, 13, 41-49. doi:10.1016/j.bspc.2014.02.009Wang, L., & Chi, Y. (2016). Blind Deconvolution From Multiple Sparse Inputs. IEEE Signal Processing Letters, 23(10), 1384-1388. doi:10.1109/lsp.2016.2599104Safont, G., Salazar, A., Vergara, L., Gomez, E., & Villanueva, V. (2018). Probabilistic Distance for Mixtures of Independent Component Analyzers. IEEE Transactions on Neural Networks and Learning Systems, 29(4), 1161-1173. doi:10.1109/tnnls.2017.2663843Safont, G., Salazar, A., Rodriguez, A., & Vergara, L. (2014). On Recovering Missing Ground Penetrating Radar Traces by Statistical Interpolation Methods. Remote Sensing, 6(8), 7546-7565. doi:10.3390/rs6087546Vergara, L., & Bernabeu, P. (2001). Simple approach to nonlinear prediction. Electronics Letters, 37(14), 926. doi:10.1049/el:20010616Ertuğrul Çelebi, M. (1997). General formula for conditional mean using higher order statistics. Electronics Letters, 33(25), 2097. doi:10.1049/el:19971432Lee, T.-W., Girolami, M., & Sejnowski, T. J. (1999). Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources. Neural Computation, 11(2), 417-441. doi:10.1162/089976699300016719Cardoso, J. F., & Souloumiac, A. (1993). Blind beamforming for non-gaussian signals. IEE Proceedings F Radar and Signal Processing, 140(6), 362. doi:10.1049/ip-f-2.1993.0054Hyvärinen, A., & Oja, E. (1997). A Fast Fixed-Point Algorithm for Independent Component Analysis. Neural Computation, 9(7), 1483-1492. doi:10.1162/neco.1997.9.7.1483Salazar, A., Vergara, L., & Miralles, R. (2010). On including sequential dependence in ICA mixture models. Signal Processing, 90(7), 2314-2318. doi:10.1016/j.sigpro.2010.02.010Lang, E. W., Tomé, A. M., Keck, I. R., Górriz-Sáez, J. M., & Puntonet, C. G. (2012). Brain Connectivity Analysis: A Short Survey. Computational Intelligence and Neuroscience, 2012, 1-21. doi:10.1155/2012/412512Fiedler, M. (1973). Algebraic connectivity of graphs. Czechoslovak Mathematical Journal, 23(2), 298-305. doi:10.21136/cmj.1973.101168Merris, R. (1994). Laplacian matrices of graphs: a survey. Linear Algebra and its Applications, 197-198, 143-176. doi:10.1016/0024-3795(94)90486-3Dong, X., Thanou, D., Frossard, P., & Vandergheynst, P. (2016). Learning Laplacian Matrix in Smooth Graph Signal Representations. IEEE Transactions on Signal Processing, 64(23), 6160-6173. doi:10.1109/tsp.2016.2602809Moragues, J., Vergara, L., & Gosalbez, J. (2011). Generalized Matched Subspace Filter for Nonindependent Noise Based on ICA. IEEE Transactions on Signal Processing, 59(7), 3430-3434. doi:10.1109/tsp.2011.2141668Egilmez, H. E., Pavez, E., & Ortega, A. (2017). Graph Learning From Data Under Laplacian and Structural Constraints. IEEE Journal of Selected Topics in Signal Processing, 11(6), 825-841. doi:10.1109/jstsp.2017.272697

Laplacian Mixture Modeling for Network Analysis and Unsupervised Learning on Graphs

Laplacian mixture models identify overlapping regions of influence in unlabeled graph and network data in a scalable and computationally efficient way, yielding useful low-dimensional representations. By combining Laplacian eigenspace and finite mixture modeling methods, they provide probabilistic or fuzzy dimensionality reductions or domain decompositions for a variety of input data types, including mixture distributions, feature vectors, and graphs or networks. Provable optimal recovery using the algorithm is analytically shown for a nontrivial class of cluster graphs. Heuristic approximations for scalable high-performance implementations are described and empirically tested. Connections to PageRank and community detection in network analysis demonstrate the wide applicability of this approach. The origins of fuzzy spectral methods, beginning with generalized heat or diffusion equations in physics, are reviewed and summarized. Comparisons to other dimensionality reduction and clustering methods for challenging unsupervised machine learning problems are also discussed.Comment: 13 figures, 35 reference

Cram\'er-Rao bounds for synchronization of rotations

Synchronization of rotations is the problem of estimating a set of rotations R_i in SO(n), i = 1, ..., N, based on noisy measurements of relative rotations R_i R_j^T. This fundamental problem has found many recent applications, most importantly in structural biology. We provide a framework to study synchronization as estimation on Riemannian manifolds for arbitrary n under a large family of noise models. The noise models we address encompass zero-mean isotropic noise, and we develop tools for Gaussian-like as well as heavy-tail types of noise in particular. As a main contribution, we derive the Cram\'er-Rao bounds of synchronization, that is, lower-bounds on the variance of unbiased estimators. We find that these bounds are structured by the pseudoinverse of the measurement graph Laplacian, where edge weights are proportional to measurement quality. We leverage this to provide interpretation in terms of random walks and visualization tools for these bounds in both the anchored and anchor-free scenarios. Similar bounds previously established were limited to rotations in the plane and Gaussian-like noise