Unveiling the frontiers of deep learning: innovations shaping diverse domains

Deep learning (DL) enables the development of computer models that are capable of learning, visualizing, optimizing, refining, and predicting data. In recent years, DL has been applied in a range of fields, including audio-visual data processing, agriculture, transportation prediction, natural language, biomedicine, disaster management, bioinformatics, drug design, genomics, face recognition, and ecology. To explore the current state of deep learning, it is necessary to investigate the latest developments and applications of deep learning in these disciplines. However, the literature is lacking in exploring the applications of deep learning in all potential sectors. This paper thus extensively investigates the potential applications of deep learning across all major fields of study as well as the associated benefits and challenges. As evidenced in the literature, DL exhibits accuracy in prediction and analysis, makes it a powerful computational tool, and has the ability to articulate itself and optimize, making it effective in processing data with no prior training. Given its independence from training data, deep learning necessitates massive amounts of data for effective analysis and processing, much like data volume. To handle the challenge of compiling huge amounts of medical, scientific, healthcare, and environmental data for use in deep learning, gated architectures like LSTMs and GRUs can be utilized. For multimodal learning, shared neurons in the neural network for all activities and specialized neurons for particular tasks are necessary.Comment: 64 pages, 3 figures, 3 table

A new method for fraud detection in credit cards based on transaction dynamics in subspaces

[EN] This paper presents a new method for fraud detection in credit cards based on exploiting the dynamics of the card transactions. We hypothesize different behavior models in the use of the card between legitimate clients and fraudsters that are registered in the sequential pattern that follows the transactions. The method considers analyses in subspaces defined by two or three variables recorded in the transactions. From these subspaces, several dynamic features, such as transaction velocity and acceleration, are estimated as input vectors for a classification process. Linear and quadratic discriminant analysis and random forest are implemented as single classifiers. All the single classification results obtained for each of the subspaces are late fused to obtain an overall result using alpha integration algorithm. The proposed method was evaluated using a subset of real data with a very low fraud to legitimate transaction ratio. We demonstrated that the temporal dependence of card transactions exploited in different subspaces and fused to give an overall result improves the detection accuracy of fraud detection in credit cards.This work was supported by Generalitat Valenciana under grant PROMETEO/2019/109, and Spanish Administration and European Union grant TEC2017-84743-P.Salazar Afanador, A.; Safont, G.; Vergara Domínguez, L. (2019). A new method for fraud detection in credit cards based on transaction dynamics in subspaces. IEEE. 722-725. https://doi.org/10.1109/CSCI49370.2019.00137S72272

A New Surrogating Algorithm by the Complex Graph Fourier Transform (CGFT)

[EN] The essential step of surrogating algorithms is phase randomizing the Fourier transform while preserving the original spectrum amplitude before computing the inverse Fourier transform. In this paper, we propose a new method which considers the graph Fourier transform. In this manner, much more flexibility is gained to define properties of the original graph signal which are to be preserved in the surrogates. The complex case is considered to allow unconstrained phase randomization in the transformed domain, hence we define a Hermitian Laplacian matrix that models the graph topology, whose eigenvectors form the basis of a complex graph Fourier transform. We have shown that the Hermitian Laplacian matrix may have negative eigenvalues. We also show in the paper that preserving the graph spectrum amplitude implies several invariances that can be controlled by the selected Hermitian Laplacian matrix. The interest of surrogating graph signals has been illustrated in the context of scarcity of instances in classifier training.This research was funded by the Spanish Administration and the European Union under grant TEC2017-84743-P.Belda, J.; Vergara Domínguez, L.; Safont Armero, G.; Salazar Afanador, A.; Parcheta, Z. (2019). A New Surrogating Algorithm by the Complex Graph Fourier Transform (CGFT). Entropy. 21(8):1-18. https://doi.org/10.3390/e21080759S118218Schreiber, T., & Schmitz, A. (2000). Surrogate time series. Physica D: Nonlinear Phenomena, 142(3-4), 346-382. doi:10.1016/s0167-2789(00)00043-9Miralles, R., Vergara, L., Salazar, A., & Igual, J. (2008). Blind detection of nonlinearities in multiple-echo ultrasonic signals. IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control, 55(3), 637-647. doi:10.1109/tuffc.2008.688Mandic, D. ., Chen, M., Gautama, T., Van Hulle, M. ., & Constantinides, A. (2008). On the characterization of the deterministic/stochastic and linear/nonlinear nature of time series. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 464(2093), 1141-1160. doi:10.1098/rspa.2007.0154Rios, R. A., Small, M., & de Mello, R. F. (2015). Testing for Linear and Nonlinear Gaussian Processes in Nonstationary Time Series. International Journal of Bifurcation and Chaos, 25(01), 1550013. doi:10.1142/s0218127415500133Borgnat, P., Flandrin, P., Honeine, P., Richard, C., & Xiao, J. (2010). Testing Stationarity With Surrogates: A Time-Frequency Approach. IEEE Transactions on Signal Processing, 58(7), 3459-3470. doi:10.1109/tsp.2010.2043971Shuman, D. I., Narang, S. K., Frossard, P., Ortega, A., & Vandergheynst, P. (2013). The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Magazine, 30(3), 83-98. doi:10.1109/msp.2012.2235192Sandryhaila, A., & Moura, J. M. F. (2013). Discrete Signal Processing on Graphs. IEEE Transactions on Signal Processing, 61(7), 1644-1656. doi:10.1109/tsp.2013.2238935Sandryhaila, A., & Moura, J. M. F. (2014). Big Data Analysis with Signal Processing on Graphs: Representation and processing of massive data sets with irregular structure. IEEE Signal Processing Magazine, 31(5), 80-90. doi:10.1109/msp.2014.2329213Pirondini, E., Vybornova, A., Coscia, M., & Van De Ville, D. (2016). A Spectral Method for Generating Surrogate Graph Signals. IEEE Signal Processing Letters, 23(9), 1275-1278. doi:10.1109/lsp.2016.2594072Sandryhaila, A., & Moura, J. M. F. (2014). Discrete Signal Processing on Graphs: Frequency Analysis. IEEE Transactions on Signal Processing, 62(12), 3042-3054. doi:10.1109/tsp.2014.2321121Shuman, D. I., Ricaud, B., & Vandergheynst, P. (2016). Vertex-frequency analysis on graphs. Applied and Computational Harmonic Analysis, 40(2), 260-291. doi:10.1016/j.acha.2015.02.005Dong, X., Thanou, D., Frossard, P., & Vandergheynst, P. (2016). Learning Laplacian Matrix in Smooth Graph Signal Representations. IEEE Transactions on Signal Processing, 64(23), 6160-6173. doi:10.1109/tsp.2016.2602809Perraudin, N., & Vandergheynst, P. (2017). Stationary Signal Processing on Graphs. IEEE Transactions on Signal Processing, 65(13), 3462-3477. doi:10.1109/tsp.2017.2690388Yu, G., & Qu, H. (2015). Hermitian Laplacian matrix and positive of mixed graphs. Applied Mathematics and Computation, 269, 70-76. doi:10.1016/j.amc.2015.07.045Gilbert, G. T. (1991). Positive Definite Matrices and Sylvester’s Criterion. The American Mathematical Monthly, 98(1), 44-46. doi:10.1080/00029890.1991.11995702Merris, R. (1994). Laplacian matrices of graphs: a survey. Linear Algebra and its Applications, 197-198, 143-176. doi:10.1016/0024-3795(94)90486-3Shapiro, H. (1991). A survey of canonical forms and invariants for unitary similarity. Linear Algebra and its Applications, 147, 101-167. doi:10.1016/0024-3795(91)90232-lFutorny, V., Horn, R. A., & Sergeichuk, V. V. (2017). Specht’s criterion for systems of linear mappings. Linear Algebra and its Applications, 519, 278-295. doi:10.1016/j.laa.2017.01.006Mazumder, R., & Hastie, T. (2012). The graphical lasso: New insights and alternatives. Electronic Journal of Statistics, 6(0), 2125-2149. doi:10.1214/12-ejs740Baba, K., Shibata, R., & Sibuya, M. (2004). PARTIAL CORRELATION AND CONDITIONAL CORRELATION AS MEASURES OF CONDITIONAL INDEPENDENCE. Australian New Zealand Journal of Statistics, 46(4), 657-664. doi:10.1111/j.1467-842x.2004.00360.xChen, X., Xu, M., & Wu, W. B. (2013). Covariance and precision matrix estimation for high-dimensional time series. The Annals of Statistics, 41(6), 2994-3021. doi:10.1214/13-aos1182Theiler, J., Eubank, S., Longtin, A., Galdrikian, B., & Doyne Farmer, J. (1992). Testing for nonlinearity in time series: the method of surrogate data. Physica D: Nonlinear Phenomena, 58(1-4), 77-94. doi:10.1016/0167-2789(92)90102-sSchreiber, T., & Schmitz, A. (1996). Improved Surrogate Data for Nonlinearity Tests. Physical Review Letters, 77(4), 635-638. doi:10.1103/physrevlett.77.635MAMMEN, E., NANDI, S., MAIWALD, T., & TIMMER, J. (2009). EFFECT OF JUMP DISCONTINUITY FOR PHASE-RANDOMIZED SURROGATE DATA TESTING. International Journal of Bifurcation and Chaos, 19(01), 403-408. doi:10.1142/s0218127409022968Lucio, J. H., Valdés, R., & Rodríguez, L. R. (2012). Improvements to surrogate data methods for nonstationary time series. Physical Review E, 85(5). doi:10.1103/physreve.85.056202Schreiber, T. (1998). Constrained Randomization of Time Series Data. Physical Review Letters, 80(10), 2105-2108. doi:10.1103/physrevlett.80.2105Prichard, D., & Theiler, J. (1994). Generating surrogate data for time series with several simultaneously measured variables. Physical Review Letters, 73(7), 951-954. doi:10.1103/physrevlett.73.951Belda, J., Vergara, L., Salazar, A., & Safont, G. (2018). Estimating the Laplacian matrix of Gaussian mixtures for signal processing on graphs. Signal Processing, 148, 241-249. doi:10.1016/j.sigpro.2018.02.017Belda, J., Vergara, L., Safont, G., & Salazar, A. (2018). Computing the Partial Correlation of ICA Models for Non-Gaussian Graph Signal Processing. Entropy, 21(1), 22. doi:10.3390/e21010022Liao, T. W. (2008). Classification of weld flaws with imbalanced class data. Expert Systems with Applications, 35(3), 1041-1052. doi:10.1016/j.eswa.2007.08.044Song, S.-J., & Shin, Y.-K. (2000). Eddy current flaw characterization in tubes by neural networks and finite element modeling. NDT & E International, 33(4), 233-243. doi:10.1016/s0963-8695(99)00046-8Bhattacharyya, S., Jha, S., Tharakunnel, K., & Westland, J. C. (2011). Data mining for credit card fraud: A comparative study. Decision Support Systems, 50(3), 602-613. doi:10.1016/j.dss.2010.08.008Mitra, S., & Acharya, T. (2007). Gesture Recognition: A Survey. IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 37(3), 311-324. doi:10.1109/tsmcc.2007.893280Dardas, N. H., & Georganas, N. D. (2011). Real-Time Hand Gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques. IEEE Transactions on Instrumentation and Measurement, 60(11), 3592-3607. doi:10.1109/tim.2011.2161140Boashash, B. (1992). Estimating and interpreting the instantaneous frequency of a signal. I. Fundamentals. Proceedings of the IEEE, 80(4), 520-538. doi:10.1109/5.135376Horn, A. (1954). Doubly Stochastic Matrices and the Diagonal of a Rotation Matrix. American Journal of Mathematics, 76(3), 620. doi:10.2307/237270

Differential evolution technique on weighted voting stacking ensemble method for credit card fraud detection

Differential Evolution is an optimization technique of stochastic search for a population-based vector, which is powerful and efficient over a continuous space for solving differentiable and non-linear optimization problems. Weighted voting stacking ensemble method is an important technique that combines various classifier models. However, selecting the appropriate weights of classifier models for the correct classification of transactions is a problem. This research study is therefore aimed at exploring whether the Differential Evolution optimization method is a good approach for defining the weighting function. Manual and random selection of weights for voting credit card transactions has previously been carried out. However, a large number of fraudulent transactions were not detected by the classifier models. Which means that a technique to overcome the weaknesses of the classifier models is required. Thus, the problem of selecting the appropriate weights was viewed as the problem of weights optimization in this study. The dataset was downloaded from the Kaggle competition data repository. Various machine learning algorithms were used to weight vote a class of transaction. The differential evolution optimization techniques was used as a weighting function. In addition, the Synthetic Minority Oversampling Technique (SMOTE) and Safe Level Synthetic Minority Oversampling Technique (SL-SMOTE) oversampling algorithms were modified to preserve the definition of SMOTE while improving the performance. Result generated from this research study showed that the Differential Evolution Optimization method is a good weighting function, which can be adopted as a systematic weight function for weight voting stacking ensemble method of various classification methods.School of ComputingM. Sc. (Computing

Tuning a variational autoencoder for data accountability problem in the Mars Science Laboratory ground data system

The Mars Curiosity rover is frequently sending back engineering and science data that goes through a pipeline of systems before reaching its final destination at the mission operations center making it prone to volume loss and data corruption. A ground data system analysis (GDSA) team is charged with the monitoring of this flow of information and the detection of anomalies in that data in order to request a re-transmission when necessary. This work presents $\Delta$-MADS, a derivative-free optimization method applied for tuning the architecture and hyperparameters of a variational autoencoder trained to detect the data with missing patches in order to assist the GDSA team in their mission

A supervised generative optimization approach for tabular data

Synthetic data generation has emerged as a crucial topic for financial institutions, driven by multiple factors, such as privacy protection and data augmentation. Many algorithms have been proposed for synthetic data generation but reaching the consensus on which method we should use for the specific data sets and use cases remains challenging. Moreover, the majority of existing approaches are ``unsupervised'' in the sense that they do not take into account the downstream task. To address these issues, this work presents a novel synthetic data generation framework. The framework integrates a supervised component tailored to the specific downstream task and employs a meta-learning approach to learn the optimal mixture distribution of existing synthetic distributions

Human-Interpretable Explanations for Black-Box Machine Learning Models: An Application to Fraud Detection

Machine Learning (ML) has been increasingly used to aid humans making high-stakes decisions in a wide range of areas, from public policy to criminal justice, education, healthcare, or financial services. However, it is very hard for humans to grasp the rationale behind every ML model’s prediction, hindering trust in the system. The field of Explainable Artificial Intelligence (XAI) emerged to tackle this problem, aiming to research and develop methods to make those “black-boxes” more interpretable, but there is still no major breakthrough. Additionally, the most popular explanation methods — LIME and SHAP — produce very low-level feature attribution explanations, being of limited usefulness to personas without any ML knowledge. This work was developed at Feedzai, a fintech company that uses ML to prevent financial crime. One of the main Feedzai products is a case management application used by fraud analysts to review suspicious financial transactions flagged by the ML models. Fraud analysts are domain experts trained to look for suspicious evidence in transactions but they do not have ML knowledge, and consequently, current XAI methods do not suit their information needs. To address this, we present JOEL, a neural network-based framework to jointly learn a decision-making task and associated domain knowledge explanations. JOEL is tailored to human-in-the-loop domain experts that lack deep technical ML knowledge, providing high-level insights about the model’s predictions that very much resemble the experts’ own reasoning. Moreover, by collecting the domain feedback from a pool of certified experts (human teaching), we promote seamless and better quality explanations. Lastly, we resort to semantic mappings between legacy expert systems and domain taxonomies to automatically annotate a bootstrap training set, overcoming the absence of concept-based human annotations. We validate JOEL empirically on a real-world fraud detection dataset, at Feedzai. We show that JOEL can generalize the explanations from the bootstrap dataset. Furthermore, obtained results indicate that human teaching is able to further improve the explanations prediction quality.A Aprendizagem de Máquina (AM) tem sido cada vez mais utilizada para ajudar os humanos a tomar decisões de alto risco numa vasta gama de áreas, desde política até à justiça criminal, educação, saúde e serviços financeiros. Porém, é muito difícil para os humanos perceber a razão da decisão do modelo de AM, prejudicando assim a confiança no sistema. O campo da Inteligência Artificial Explicável (IAE) surgiu para enfrentar este problema, visando desenvolver métodos para tornar as “caixas-pretas” mais interpretáveis, embora ainda sem grande avanço. Além disso, os métodos de explicação mais populares — LIME and SHAP — produzem explicações de muito baixo nível, sendo de utilidade limitada para pessoas sem conhecimento de AM. Este trabalho foi desenvolvido na Feedzai, a fintech que usa a AM para prevenir crimes financeiros. Um dos produtos da Feedzai é uma aplicação de gestão de casos, usada por analistas de fraude. Estes são especialistas no domínio treinados para procurar evidências suspeitas em transações financeiras, contudo não tendo o conhecimento em AM, os métodos de IAE atuais não satisfazem as suas necessidades de informação. Para resolver isso, apresentamos JOEL, a framework baseada em rede neuronal para aprender conjuntamente a tarefa de tomada de decisão e as explicações associadas. A JOEL é orientada a especialistas de domínio que não têm conhecimento técnico profundo de AM, fornecendo informações de alto nível sobre as previsões do modelo, que muito se assemelham ao raciocínio dos próprios especialistas. Ademais, ao recolher o feedback de especialistas certificados (ensino humano), promovemos explicações contínuas e de melhor qualidade. Por último, recorremos a mapeamentos semânticos entre sistemas legados e taxonomias de domínio para anotar automaticamente um conjunto de dados, superando a ausência de anotações humanas baseadas em conceitos. Validamos a JOEL empiricamente em um conjunto de dados de detecção de fraude do mundo real, na Feedzai. Mostramos que a JOEL pode generalizar as explicações aprendidas no conjunto de dados inicial e que o ensino humano é capaz de melhorar a qualidade da previsão das explicações