Non-negative matrix decomposition for single-channel source separation in biomedical signal processing applications

La separación de fuentes en el procesamiento de señales digitales, consiste en encontrar las mejores aproximaciones de las componentes de una mezcla de señales. Aunque en la mayoría de los casos no se dispone de antemano de una información detallada sobre las fuentes, es posible realizar una separación parcial. Uno de los posibles métodos es la factorización de matrices no negativa (NMF). A pesar de su creciente popularidad en la comunidad de procesamiento de señales biomédicas, se presta poca atención a los importantes inconvenientes que a menudo impiden su uso de forma directa. Uno de estos inconvenientes es la inicialización aleatoria del algoritmo, lo que a menudo lleva a un mínimo local y a resultados irreproducibles. La selección del rango de las fuentes individuales es a menudo engañosa. Una solución habitual para este problema es asignar el rango de acuerdo con la cantidad de fuentes y luego ajustarlo mediante un procedimiento iterativo de prueba y error. Desafortunadamente, este procedimiento es computacionalmente costoso y no hay garantía de que converja al rango óptimo para cada fuente. Otro aspecto importante es la transformación utilizada para pasar del dominio del tiempo a la representación no negativa (matricial) y viceversa. En la presente tesis se abordan los problemas mencionados y se proponen nuevas características para algoritmo, tales como: estimación inequívoca del rango no-negativo e inicialización con estructuras cuidadosamente diseñadas. Todos los métodos propuestos se han comparado con al menos dos de referencia.Source separation in digital signal processing consists of finding best estimates of the signals involved in a signal mixture. Although, in most cases a detailed information about the sources is not known in advance, a partial separation is still possible. One of possible methods is non-negative matrix factorization NMF. In spite of its increasing popularity in the biomedical signal processing community, a little attention is paid to its serious drawbacks which often make impossible the straightforward use of the available “off-shelf” algorithm. One of them is a random initialization of an algorithm what often leads to a local minimum and irreproducible results. The selection of the non-negative rank of individual sources is often misleading. A usual shortcut to this problem is to assign rank according to the number of sources and then to tune it up by some iterative trial-and-error input matrix decomposition procedure. Such an approach is computationally costly and is not guaranteed to converge to optimal rank for each source. Moreover, a synthesis of time-domain waveforms from the low-rank source descriptions is often hard, due to the fact that the original phases are unknown. In the present thesis we address the aforementioned drawbacks and introduce new algorithm features, namely: unambiguous non-negative rank estimation and initialization with carefully designed structures. All proposed methods have been compared to at least two-state-of art reference methods.Programa de Doctorado en Tecnologías de las Comunicaciones, Bioingeniería y de las Energías Renovables (RD 99/2011)Bioingeniaritzako eta Komunikazioen eta Energia Berriztagarrien Teknologietako Doktoretza Programa (ED 99/2011

Fault diagnosis in industrial process by using LSTM and an elastic net

[EN] Fault diagnosis is important for industrial processes because it permits to determine the necessity of emergency stops in a process and/or to propose a maintenance plan. Two strategies for fault diagnosis are compared in this work. On the one hand, the data are preprocessed using the independent components analysis for dimension reduction, then the wavelet transform is used in order to highlight the faulty signals, with this information an artificial neural network was fed. On the other hand, the second strategy, the main contribution of this work, is the implementation of a long short term memory. This memory is fed with the most representative variables selected by an elastic net to use both, the L1 and L2 norms. These strategies are applied in the Tennessee Eastman process, a benchmark widely used for fault diagnosis. The fault isolation had better results than those reported in the literature.[ES] El diagnóstico de fallas es importante en los procesos industriales, ya que permite determinar si es necesario detener el proceso en operación y/o proponer un plan de mantenimiento. En el presente trabajo se comparan dos estrategias para diagnosticar fallas. La primera realiza un preprocesamiento de datos usando el análisis de componentes independientes para reducir la dimensión de los datos, posteriormente, se emplea la transformada wavelet para resaltar las señales de falla, con esta información se alimenta una red neuronal artificial. Por su parte, la segunda estrategia, principal contribución de este trabajo, usa una memoria de corto y largo plazo. Esta memoria es alimentada por las variables más significativas seleccionadas mediante una red elástica para usar tanto la norma $L_1$ como la $L_2$. Como ejemplo de aplicación se utilizó el proceso químico Tennessee Eastman, un proceso ampliamente usado en el diagnóstico de fallas. El aislamiento de fallas mostró mejores resultados con respecto a los reportados en la literatura.Márquez-Vera, MA.; López-Ortega, O.; Ramos-Velasco, LE.; Ortega-Mendoza, RM.; Fernández-Neri, BJ.; Zúñiga-Peña, NS. (2021). Diagnóstico de fallas mediante una LSTM y una red elástica. Revista Iberoamericana de Automática e Informática industrial. 18(2):164-175. https://doi.org/10.4995/riai.2020.13611OJS164175182Adewole, A., Tzoneva, R., Behardien, S., 2016. Distribution network fault section identification and fault location using wavelet entropy and neural networks. Applied Soft Computing 46, 296-306. https://doi.org/10.1016/j.asoc.2016.05.013Alkaya, A., Eker, I., 2011. Variance sensitive adaptive threshold-based PCA method for fault detection with experimental application. ISA Transactions 50, 287-302. https://doi.org/10.1016/j.isatra.2010.12.004Barakat, S., Eteiba, M., Wahba, W., 2014. Fault location in underground cables using anfis nets and discrete wavelet transform. Journal of Electrical Systems and Information Technology 1, 198-211. https://doi.org/10.1016/j.jesit.2014.12.003Bathelt, A., Ricker, N., Jelali, M., 2015. Revision of the Tennessee Eastman process model. IFAC Papers-Online 48 (8), 309-314. https://doi.org/10.1016/j.ifacol.2015.08.199Boldt, F., Rauber, T., Varejao, F., October 2014. Evaluation of the extreme learning machine for automatic fault diagnosis of the Tennessee Eastman chemical process. In: IEEE (Ed.), Annual Conference of the IEEE Industrial Electronics Society. Vol. 40. Dallas, Texas, pp. 2551-2557. https://doi.org/10.1109/IECON.2014.7048865Chen, H., Tino, P., Yao, X., 2014. Cognitive fault diagnosis in Tennessee Eastman process using learning in the model space. Computers and Chemical Engineering 67, 33-42. https://doi.org/10.1016/j.compchemeng.2014.03.015Rodrigues, J., Filho, P., PeixotoJr., E., Kumar, A., deAlbuquerque, V., 2019. Classification of EEG signals to detect alcoholism using machine learning techniques. Pattern Recognition Letters 125, 140-149. https://doi.org/10.1016/j.patrec.2019.04.019Dixit, A., Majumdar, S., 2013. Comparative analysis of coiflet and daubechies wavelets using global threshold for image denoising. Intenational Journal of Advances in Engineering & Technology 6 (5), 2247-2252.Downs, J., Vogel, E., 1993. A plant-wide industrial process control problem. Computers and Chemical Engineering 17 (3), 245-255. https://doi.org/10.1016/0098-1354(93)80018-IFischer, T., Krauss, C., 2018. Deep learning with long short-term memory networks for financial market predictions. European Journal of Operational Research 270, 654-669. https://doi.org/10.1016/j.ejor.2017.11.054Gao, X., Hou, J., 2016. An improved SVM integrated GS-PCA fault diagnosis approach of Tennessee Eastman process. Neurocomputing 174, 906-911. https://doi.org/10.1016/j.neucom.2015.10.018Geng, Z., Li, Z., Han, Y., 2018. A new deep belief network based on RBM with glial chains. Information Sciences 463, 294-306. https://doi.org/10.1016/j.ins.2018.06.043Goodfellow, I., Bengio, Y., Courville, A., 2016. Deep Learning. MIT Press, United States of America, http://www.deeplearningbook.ogr.Han, L., Li, C., Guo, S., Su, X., 2015. Feature extraction method of bearing AE signal based on improved Fast-ICA and wavelet packet energy. Mechanical Systems and Signal Processing 62-63, 91-99. https://doi.org/10.1016/j.ymssp.2015.03.009Hastie, T., Tibshirani, R., Friedman, J., 2009. The elements of statistical learning: data mining, inference and prediction. Springer, New York. https://doi.org/10.1007/978-0-387-84858-7Hoang, D., Kang, H., 2019. A survey on deep learning based bearing fault diagnosis. Neurocomputing 335, 327-335. ttps://doi.org/10.1016/j.neucom.2018.06.078Hochreiter, S., Schmidhuber, J., 1997. Long short term memory. Neural Computation 9 (8), 1735-1780. ttps://doi.org/10.1162/neco.1997.9.8.1735Hyvärinen, A., Oja, E., 2000. Independent component analysis: Algorithms and applications. Neural Networks 13, 411-430. ttps://doi.org/10.1016/S0893-6080(00)00026-5Jing, C., Gao, X., Zhu, X., Lang, S., July 2014. Fault classificaction on Tennessee Eastman process: PCA and SVM. In: IEEE (Ed.), Intenational Conference on Mecatronics and Control. Jinzhou, China, pp. 2194-2197. https://doi.org/10.1109/ICMC.2014.7231958Jung, C., Kim, K., Lee, J., Klockl, B., 2007. Wavelet and neuro-fuzzy based fault location for combined transmission systems. Energy Systems 29, 445-454. https://doi.org/10.1016/j.ijepes.2006.11.003Kandula, V. K., 2011. Fault detection in process control plants using principal component analysis. Master's thesis, Louisiana State University, Department of Electrical Engineering.Karpenko, M., Sepehri, N., Octubre 2001. A neural network based fault detection and identification scheme for pneumatic process control valves. In: IEEE (Ed.), International Conference on Systems, Man and Cybernetics. Tucson, USA, pp. 93-98. https://doi.org/10.1109/ICSMC.2001.969794Khakipour, M., Safavi, A., Setoodeh, P., 2017. Bearing fault diagnosis with morphological gradient wavelet. Journal of the Franklin Institute 354, 2465-2476. https://doi.org/10.1016/j.jfranklin.2016.11.013Kuang, T., Yang, Z., Yao, Y., 2015. Multivariate fault isolation via variable selection in discriminant analysis. Journal of Process Control 35, 30-40. https://doi.org/10.1016/j.isatra.2017.06.014Kumar, R., Bansal, H., 2019. Hardware in the loop implementation of wavelet based strategy in shuntactive powerfilter to mitigate power quality issues. Electric Power Systems Research 169, 92-104. https://doi.org/10.1016/j.epsr.2019.01.001Lau, C., Ghosh, K., Hussain, M., Hassan, C. C., 2013. Fault disgnosis of Tennessee Eastman process with multi-scale PCA and ANFIS. Chemom. Intell. Lab. Syst. 120, 1-14. https://doi.org/10.1016/j.chemolab.2012.10.005Lee, J., Yoo, C., Lee, I., 2004. Statistical process monitoring with independent component analysis. Journal of Process Control 14 (5), 467-485. https://doi.org/10.1016/j.jprocont.2003.09.004Lei, J., Liu, C., Jiang, D., 2019. Fault diagnosis of wind turbine based on long short-term memory networks. Renewable Energy 133, 422-432. https://doi.org/10.1016/j.renene.2018.10.031Li, W., Monti, A., Ponci, F., 2014. Fault detection and classification in medium voltage DC shipboard power systems with wavelets and artificial neural networks. IEEE Transactions on Instrumentation and Measurement 63 (11), 2651-2665. https://doi.org/10.1109/TIM.2014.2313035Liang, P., Deng, C.,Wu, J., Yang, Z., Zhu, J., Zhang, Z., 2019. Compound fault diagnosis of gearboxes via multi-label convolutional neural network and wavelet transform. Computers in Industry 113, 103132. https://doi.org/10.1016/j.compind.2019.103132Lin, J., Zhang, A., 2005. Fault feature separation using wavelet-ICA filter. NDT&E International 38, 421-427. https://doi.org/10.1016/j.ndteint.2004.11.005Linker, R., Gutman, P., Seginer, I., 2002. Observer-based robust failure detection and isolation in greenhouses. Control Engineering Practice 10 (5), 519- 531. https://doi.org/10.1016/S0967-0661(02)00002-3Lou, W., Loparo, K., 2004. Bearing fault diagnosis based on wavelet transform and fuzzy inference. Mechanical Systems and Signal Processing 18, 1077-1095. https://doi.org/10.1016/S0888-3270(03)00077-3Lv, F.,Wen, C., Bao, Z., Liu, M., 2016. Fault diagnosis based on deep learning. In: AACC (Ed.), American Control Conference. Boston, USA, pp. 6851-6856. https://doi.org/10.1109/ACC.2016.7526751Lv, F., Wen, C., Liu, M., Bao, Z., 2017. Weighted time series fault diagnosis based on a staked sparce autoencoder. Journal of Chemometrics 31, 16 pages. https://doi.org/10.1002/cem.2912Lv, F., Fan, X., Wen, C., Bao, Z., 2018. Stacked sparse auto encoder network based multimode process monitoring. In: IEEE (Ed.), International Conference on Control Automation & Information Science. Hangzhou, China, pp. 227-232. https://doi.org/10.1109/ICCAIS.2018.8570618Maglaveras, N., Stamkopoulos, T., Diamantaras, K., Pappas, C., Strintzis, M., 1998. ECG pattern recognition and classification using non-linear transfor mations and neural networks: A review. International Journal of Medical Informatics 52, 191-208. https://doi.org/10.1016/S1386-5056(98)00138-5Methnani, S., Lafont, F., Gautier, J., Damak, T., Toumi, A., 2013. Actuator and sensor fault detection, isolation and identification in nonlinear dynamical systems, with applications to a waste water treatment plant. Journal of Computer Engineering and Informatics 1 (4), 112-125. https://doi.org/10.1080/21642583.2014.888525Muñoz-Cobo, J., Mendizábal, R., Miquel, A., Berna, C., Escrivá, A., 2017. Use of the principles of maximum entropy and maximum relative entropy for the determination of uncertain parameter distributions in engineering applications. Entropy 19, 486, 37 pages. https://doi.org/10.3390/e19090486Nguyen, B., Quyen, A., Nguyen, P., Ton, T., July 2017. Wavelet-based neural network for recognition of faults at nhabe power substation of the vietnam power system. In: IEEE (Ed.), International Conference on System Science and Engineering. Ho Chi Minh City, Vietnam, pp. 165-168. https://doi.org/10.1109/ICSSE.2017.8030858Ojeda-González, A., Mendes-Jr., O., Oliveira-Domingues, M., Menconi, V., 2014. Daubechies wavelet coeffcients: a tool to study interplanetary magnetic field fluctuations. Geof'ısica Internacional 53 (2), 101-115. https://doi.org/10.1016/S0016-7169(14)71494-1Oliveira, J., Pontes, K., Santori, I., Embirucu, M., 2017. Fault detection and diagnosis in dynamic systems using weightless neural networks. Expert Systems With Applications 84, 200-219. https://doi.org/10.1016/j.eswa.2017.05.020Patan, K., 2008. Artificial neural networks for the modelling and fault diagnosis of technical process. Lecture Notes in Control and Information Sciences. Springer, India.Rafiee, J., Rafiee, M., Tse, P., 2010. Application of mother wavelet functions for automatic gear and bearing fault diagnosis. Expert Systems with Applications 37, 4568-4579. https://doi.org/10.1016/j.eswa.2009.12.051Ramos-Velasco, L., Ramos-Fernández, J., Islar-Gómez, O., Espejel-Rivera, M., García-Lamont, J., Márquez-Vera, M., 2013. Identificación y control wavenet de un motor de ca. Revista Iberoamericana de Automática e Informática Industrial 10, 269-278. https://doi.org/10.1016/j.riai.2013.05.002Rato, T., Reis, M., 2013. Defining the structure of DPCA models and its impact on process monitoring and prediction ctivities. Chemometrics and Intelligent Laboratory Systems 125, 74-86. https://doi.org/10.1016/j.chemolab.2013.03.009Rockinger, M., Jondeau, E., 2002. Entropy densities with an application to autoregressive conditional skewness and kurtosis. Journal of Econometrics 106, 119-142. https://doi.org/10.1016/S0304-4076(01)00092-6Salahschoor, K., Kiasi, F., July 2008. On-line process monitoring based on wavelet-ICA methodology. In: IFAC (Ed.), Proceedings of the 17th World Congress. Seul- Korea, pp. 6-11. https://doi.org/10.3182/20080706-5-KR-1001.01253Salahshoor, K., Khoshro, M., Kordestani, M., 2011. Fault detection and diagnosis of an industrial steam turbine using a distributed configuration of adaptive neuro-fuzzy inference systems. Simulation Modelling Practice and Theory 19, 1280-1293. https://doi.org/10.1016/j.simpat.2011.01.005Sharif, I., Khare, S., 2014. Comparative analysis of Haar and Daubechies wavelet for hyper spectral image classification. In: Commission, I. T. (Ed.), VIII Symposium of The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Science. Hyderabad-India, pp. 937-941. https://doi.org/10.5194/isprsarchives-XL-8-937-2014Smirnov, E., Timoshenko, D., Adrianov, S., 2014. Comparison of regularization methods for imagenet classification with deep convolutional neural networks. AASRI Procedia 6, 89-94. https://doi.org/10.1016/j.aasri.2014.05.013Sobhani-Tehrani, E., Khorasani, K., 2009. Fault diagnosis of nonlinear systems using a hybrid approach. Fault detetion and diagnosis. Springer, Berlin, Ch. 2, pp. 22-49. https://doi.org/10.1007/978-0-387-92907-1_2Tayarani-Bathaie, S., Vanini, Z., Khorasani, K., 2014. Dynamic neural networkbased fault diagnosis of gas turbine engines. Neurocomputing 125, 153-165. https://doi.org/10.1016/j.neucom.2012.06.050Zvokelj, M., Zupan, S., Prebil, I., 2016. EEMD-based multiscale ICA method for slewing bearing fault detection and diagnosis. Journal of Sound and Vibration 26, 394-423. https://doi.org/10.1016/j.jsv.2016.01.046Wang, X., Qin, Y., Wang, Y., Xiang, S., Chen, H., 2019. ReLTanh: An activation function with vanishing gradient resistance for SAE-based DNNs and its application to rotating machinery fault diagnosis. Neurocomputing 363, 88-98. https://doi.org/10.1016/j.neucom.2019.07.017Wu, F., Tong, F., Yang, Z., 2016. EMGdi signal enhancement based on ICA decomposition and wavelet transform. Applied Soft Computing 43, 561-571. https://doi.org/10.1016/j.asoc.2016.03.002Wu, J., Hsu, C., Wu, G., 2009. Fault gear identification and classification using discrete wavelet transform and adaptive neuro-fuzzy inference. Expert Systems with Applications 36, 6244-6255. https://doi.org/10.1016/j.eswa.2008.07.023Wu, Q., Law, R., Wu, S., 2011. Fault diagnosis of car assembly line based on fuzzy wavelet kernel support vector classifier machine and modified genetic algorithm. Expert Systems with Applications 38, 9096-9104. https://doi.org/10.1016/j.eswa.2010.12.109Wu, H., Zhao., Jinsong, 2018. Deep convolutional neural network model based chemical process fault diagnosis. Computers and Chemical Engineering 115, 185-197. https://doi.org/10.1016/j.compchemeng.2018.04.009Xiao, C., Chen, N., Hu, C., Wang, K., Gong, J., Chen, Z., 2019. Short and midterm sea surface temperature prediction using time-series satellite data and LSTM-AdaBoost combination approach. Remote Sensing of Environment 233, 111358. https://doi.org/10.1016/j.rse.2019.111358Xie, D., Bai, L., December 2015. A hierarchical deep neural network for fault diagnosis on Tennessee-Eastman process. In: IEEE (Ed.), International Conference on Machine Learning and Applications. Vol. 14. Miami, USA, pp. 745-748. https://doi.org/10.1109/ICMLA.2015.208Yan, R., Gao, R., Chen, X., 2014. Wavelets for fault diagnosis of rotary machines: A review with applications. Signal Processing 351, 4555-4569. https://doi.org/10.1016/j.sigpro.2013.04.015Yan, Z., Yao, Y., 2015. Variable selection method for fault isolation using least absolute shrinkage and selection operator (LASSO). Chemometrics and Intelligent Laboratory Systems 146, 136-146. https://doi.org/10.1016/j.chemolab.2015.05.019Yao, G., Lei, T., Zhong, J., 2019. A review of convolutional-neural-networkbased action recognition. Pattern Recognition Letters 118, 14-22. https://doi.org/10.1016/j.patrec.2018.05.018Yin, S., Ding, S., Haghani, A., Hao, H., Zhang, P., 2012. A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process. Journal of Process Control 22, 1567-1581. https://doi.org/10.1016/j.jprocont.2012.06.009Zhang, Q., Yang, L., Chen, Z., Li, P., 2018. A survey on deep learning for big data. Information Fusion 42, 146-157. https://doi.org/10.1016/j.inffus.2017.10.006Zhang, X., Polycarpou, M., Parisini, T., 2002. A robust detection and isolation scheme for abrupt and incipient faults in nonlinear systems. IEEE Transactions on Automatic Control 47 (4), 576-593. https://doi.org/10.1109/9.995036Zhang, Y., Zhang, L., Zhang, H., 2012. Fault detection for industrial processes. Mathematical Problems in Engineering 2012, 18 pages. https://doi.org/10.1155/2012/757828Zhang, Z., Zhao, J., 2017. A deep belief network based fault diagnosis model for complex chemical process. Computers and Chemical Engineering 107, 395-407. https://doi.org/10.1016/j.compchemeng.2017.02.041Zhao, H., 2018. Neural component analysis for fault detection. Chemometrics and Intelligent Laboratory Systems 176, 11-21. https://doi.org/10.1016/j.chemolab.2018.02.001Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., Gao, R., 2019. Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing 115, 213-237. https://doi.org/10.1016/j.ymssp.2018.05.050Zheng, J., Huang, W., Wang, Z., Liang, J., 2019. Mutual information-based sparse multiblock dissimilarity method for incipient fault detection and diagnosis in plant-wide process. Journal of Process Control 83, 63-76. https://doi.org/10.1016/j.jprocont.2019.09.00