2022 SDSU Data Science Symposium Program

https://openprairie.sdstate.edu/ds_symposium_programs/1003/thumbnail.jp

Development of a methodology for the diagnosis of internal combustion engines using non-invasive measurements based on the use of interpretable neural networks applicable to databases with multiple annotators

Pressure is one of the essential variables that give information for engine condition and monitoring. Direct recording of this signal is complex and invasive, while the angular velocity can be measured easily. Nonetheless, the challenge is to predict the cylinder pressure using the shaft kinematics accurately. On the other hand, the increasing popularity of crowdsourcing platforms, i.e., Amazon Mechanical Turk, changes how datasets for supervised learning are built. In these cases, instead of having datasets labeled by one source (which is supposed to be an expert who provided the absolute gold standard), databases holding multiple annotators are provided. However, most state-of-the-art methods devoted to learning from multiple experts assume that the labeler's behavior is homogeneous across the input feature space. Besides, independence constraints are imposed on annotators' outputs. This document presents a Regularized Chained Deep Neural Network to deal with classification tasks from multiple annotators. In this thesis, we develop 2 strategies aiming to avoid intrusive techniques that are commonly used to diagnose Internal Combustion Engines (ICE). The first consist of a time-delay neural network (TDNN), interpreted as a finite pulse response (FIR) filter to estimate the in-cylinder pressure of a single-cylinder ICE from fluctuations in shaft angular velocity. The experiments are conducted over data obtained from an ICE operating in 12 different states by changing the angular velocity and load. The TDNN's delay is adjusted to get the highest possible correlation-based score. Our methodology can predict pressure with an R2>0.9, avoiding complicated pre-processing steps. The second technique, termed RCDNN, jointly predicts the ground truth label and the annotators' performance from input space samples. In turn, RCDNN codes interdependencies among the experts by analyzing the layers' weights and includes l1, l2, and Monte-Carlo Dropout-based regularizers to deal with the overfitting issue in deep learning models. Obtained results (using both simulated and real-world annotators) demonstrate that RCDNN can deal with multi-labelers scenarios for classification tasks, defeating state-of-the-art techniques.La presión es una de las variables esenciales que dan información para el estado del motor y su monitorización. El registro directo de esta señal es complejo e invasivo, mientras que la velocidad angular puede medirse fácilmente. No obstante, el reto consiste en predecir la presión del cilindro utilizando la cinemática del eje con precisión. Por otro lado, la creciente popularidad de las plataformas de crowdsourcing, por ejemplo, Amazon Mechanical Turk, cambia la forma de construir conjuntos de datos para el aprendizaje supervisado. En estos casos, en lugar de tener conjuntos de datos etiquetados por una sola fuente (que se supone que es un experto que proporcionó el estándar de oro absoluto), se proporcionan bases de datos con múltiples anotadores. Sin embargo, la mayoría de los métodos de vanguardia dedicados al aprendizaje a partir de múltiples expertos suponen que el comportamiento del etiquetador es homogéneo en todo el espacio de características de entrada. Además, se imponen restricciones de independencia a los resultados de los anotadores. Este documento presenta una Red Neuronal Profunda Encadenada Regularizada para abordar tareas de clasificación a partir de múltiples anotadores. En esta tesis, desarrollamos dos estrategias con el objetivo de evitar las técnicas intrusivas que se utilizan habitualmente para diagnosticar motores de combustión interna (ICE). La primera consiste en una red neuronal de retardo temporal (TDNN), interpretada como un filtro de respuesta de pulso finito (FIR) para estimar la presión en el cilindro de un ICE de un solo cilindro a partir de las fluctuaciones de la velocidad angular del eje. Los experimentos se realizan sobre datos obtenidos de un ICE que opera en 12 estados diferentes cambiando la velocidad angular y la carga. El retardo de la TDNN se ajusta para obtener la mayor puntuación posible basada en la correlación. Nuestra metodología puede predecir la presión con un R2>0,9, evitando complicados pasos de preprocesamiento.MaestríaMagíster en Ingeniería EléctricaContent 1 Introduction 10 1.1 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2 Justification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.3 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.3.1 General objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.3.2 Specific objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2 TDNN-based Engine In-cylinder Pressure Estimation from Shaft Velocity Spectral Representation 18 2.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2.1 Time Delay Neural Network fundamentals . . . . . . . . . . . . . . . 19 2.2.2 Harmonic prediction performance based on Magnitude-Squared Coherence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.3 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.3.1 Engine Measurements, Data Acquisition, and Preprocessing . . . . . 22 2.3.2 Pressure signal estimation . . . . . . . . . . . . . . . . . . . . . . . . 26 2.4 Results and discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.5 Conclusions and future work . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3 Master Thesis: Content 3 Regularized Chained Deep Neural Network Classifier for Multiple Annotators 37 3.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.2 Materials and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.2.1 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.2.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.3 Experimental set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.3.1 Tested datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.3.2 RCDNN detailed architecture and training . . . . . . . . . . . . . . . 46 3.3.3 Results and discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.3.4 Introducing spammers and malicious annotators . . . . . . . . . . . . 55 3.3.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4 Final Remarks 58 4.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.1.1 TDNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.1.2 RCDNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.2.1 TDNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.2.2 RCDNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.3 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

A Knowledge Base Completion Model Based on Path Feature Learning

Large-scale knowledge bases, as the foundations for promoting the development of artificial intelligence, have attracted increasing attention in recent years. These knowledge bases contain billions of facts in triple format; yet, they suffer from sparse relations between entities. Researchers proposed the path ranking algorithm (PRA) to solve this fatal problem. To improve the scalability of knowledge inference, PRA exploits random walks to find Horn clauses with chain structures to predict new relations given existing facts. This method can be regarded as a statistical classification issue for statistical relational learning (SRL). However, large-scale knowledge base completion demands superior accuracy and scalability. In this paper, we propose the path feature learning model (PFLM) to achieve this urgent task. More precisely, we define a two-stage model: the first stage aims to learn path features from the existing knowledge base and extra parsed corpus; the second stage uses these path features to predict new relations. The experimental results demonstrate that the PFLM can learn meaningful features and can achieve significant and consistent improvements compared with previous work

Pathological Brain Detection Using Weiner Filtering, 2D-Discrete Wavelet Transform, Probabilistic PCA, and Random Subspace Ensemble Classifier

Accurate diagnosis of pathological brain images is important for patient care, particularly in the early phase of the disease. Although numerous studies have used machine-learning techniques for the computer-aided diagnosis (CAD) of pathological brain, previous methods encountered challenges in terms of the diagnostic efficiency owing to deficiencies in the choice of proper filtering techniques, neuroimaging biomarkers, and limited learning models. Magnetic resonance imaging (MRI) is capable of providing enhanced information regarding the soft tissues, and therefore MR images are included in the proposed approach. In this study, we propose a new model that includes Wiener filtering for noise reduction, 2D-discrete wavelet transform (2D-DWT) for feature extraction, probabilistic principal component analysis (PPCA) for dimensionality reduction, and a random subspace ensemble (RSE) classifier along with the K-nearest neighbors (KNN) algorithm as a base classifier to classify brain images as pathological or normal ones. The proposed methods provide a significant improvement in classification results when compared to other studies. Based on 5×5 cross-validation (CV), the proposed method outperforms 21 state-of-the-art algorithms in terms of classification accuracy, sensitivity, and specificity for all four datasets used in the study

Phenotypic Characterization with Software Development for Analysis of the Visual System in Animal Models of Neurodevelopmental Diseases

A neurofibromatose tipo 1 (NF1) é uma perturbação do desenvolvimento neurológico com implicações cognitivas adultas. Provoca anomalias do sistema nervoso central e afeta 1 em 3000 indivíduos em todo o mundo. Contudo, pouco se sabe sobre os efeitos no sistema visual e como estes podem estar associados a défices cognitivos e preveem a sua progressão. Neste trabalho, avalia-se as potenciais alterações na fisiologia da retina num modelo genético de murgalho de NF1, utilizando uma técnica neurofisiológica não invasiva, o eletroretinograma (ERG), para determinar o seu potencial diagnóstico. Como um indicador fiável da função da retina em resposta à luz, o ERG tem a capacidade de ajudar a nossa interpretação da fisiopatologia das perturbações do neurodesenvolvimento e neurodegenerativas. Os principais objetivos desta tese são a caracterização fenotípica do sistema visual num modelo animal de NF1 e o desenvolvimento de ferramentas informáticas (MATLAB e Phyton) para processamento de sinais, análise de forma de onda, extração de características, e classificação. Verificou-se que os parâmetros ERG relacionados principalmente com a atividade oscilatória inibitória revelam alterações subtis dependentes do sexo. Para vários potenciais oscilatórios, machos e fêmeas exibem alterações opostas associadas ao genótipo mutante. Além disso, as características do ERG foram utilizadas para formar um classificador de aprendizagem de máquina baseado nos aglomerados significativos encontrados para algumas interações entre indivíduos, um classificador que se destina a ser capaz de receber um sinal e devolver o provável diagnóstico.Neurofibromatosis type 1 (NF1) is a neurodevelopmental disorder with adult cognitive implications. It causes central nervous system anomalies and affects 1 in 3000 individuals worldwide. However, little is known about the effects on the visual system circuitry and how these may be associated with cognitive deficits and predicts its progression. In this work, it was evaluated the potential alterations in retinal physiology in a genetic mouse model of NF1, using a non-invasive neurophysiological technique, the electroretinogram (ERG), to ascertain its diagnostic potential. As a reliable indicator of retinal function in response to light, the ERG has the ability to aid our interpretation of the pathophysiology of neurodevelopmental and neurodegenerative disorders. The main objectives of this thesis are the phenotypic characterization of the visual system in an animal model of NF1 and the development of computer tools (MATLAB and Phyton) for signal processing, waveform analysis, feature extraction, and classification. This work found that ERG parameters mainly related to inhibitory oscillatory activity reveal subtle sex-dependent alterations. For various oscillatory potentials males and females exhibit opposite changes associated with the transgenic background. Furthermore, the ERG features were used to form a machine learning classifier based on the significant clusters found for some interactions between individuals, a classifier that is meant to be able to receive a signal and return the likely diagnosis

Intelligent Devices for IoT Applications

Internet of Things (IoT) devices refer to a vast network of physical devices that are connected to the internet and can communicate with each other through sensors and software. These devices range from simple household appliances, like smart thermostats and security cameras, to more complex industrial equipment, such as sensors used in manufacturing and logistics. Specially, IoT enabled wireless gas sensing systems which can withstand harsh environments without compromising the performance are getting popular day by day, which necessitates adequate developments in this field. By being the essential components of a wireless gas sensing system, both the sensor and the elements for communication should be agile and resilient when it comes to tackle unfavorable scenario. Moreover, gas sensors are prone to drift, which can lead to inaccurate readings and decreased reliability over time. Again, recent advancements in antenna design, such as fractal antennas and metamaterial structures, have shown promises in improving the bandwidth and gain parameters of the antennas built on top of high temperature tackling substrates. This piece of research targets three fundamental sections: demonstration of recent advances in data driven techniques for gas sensing system optimization, designing of antennas for different applications, and device design as well as fabrication. The Dimatix DMP-2831 inkjet printer has been optimized to operate with six different inks and two different substrates including PET and 3 mol yttria-stabilized zirconia (3YSZ) based ceramic substrate. Later, the feature oriented gas sensor data analysis to investigate correlations among stability, selectivity and long term drift is illustrated, which should significant relations among those parameters that can be considered while designing different intelligent data driven models to compensate drift. Moreover, a subspace transfer based approach is proposed to classify drifted gas sensor response to detect particular gas with higher accuracy. The model achieved an average accuracy greater than 87% while using only 40% of the total dataset to be trained. In the field of antenna technology, a co-planar waveguide (CPW) fed super wideband antenna is proposed which can cover C, X, Ku, K, Ka, Q, V, and W bands according to the simulated performance with high gain and radiation efficiency. Again, a high temperature tolerant antenna based on 3YSZ substrate is proposed which achieved good alignment between the simulated and fabricated device performance

Applications

Volume 3 describes how resource-aware machine learning methods and techniques are used to successfully solve real-world problems. The book provides numerous specific application examples: in health and medicine for risk modelling, diagnosis, and treatment selection for diseases in electronics, steel production and milling for quality control during manufacturing processes in traffic, logistics for smart cities and for mobile communications