Foundational Models for Fault Diagnosis of Electrical Motors

A majority of recent advancements related to the fault diagnosis of electrical motors are based on the assumption that training and testing data are drawn from the same distribution. However, the data distribution can vary across different operating conditions during real-world operating scenarios of electrical motors. Consequently, this assumption limits the practical implementation of existing studies for fault diagnosis, as they rely on fully labelled training data spanning all operating conditions and assume a consistent distribution. This is because obtaining a large number of labelled samples for several machines across different fault cases and operating scenarios may be unfeasible. In order to overcome the aforementioned limitations, this work proposes a framework to develop a foundational model for fault diagnosis of electrical motors. It involves building a neural network-based backbone to learn high-level features using self-supervised learning, and then fine-tuning the backbone to achieve specific objectives. The primary advantage of such an approach is that the backbone can be fine-tuned to achieve a wide variety of target tasks using very less amount of training data as compared to traditional supervised learning methodologies. The empirical evaluation demonstrates the effectiveness of the proposed approach by obtaining more than 90\% classification accuracy by fine-tuning the backbone not only across different types of fault scenarios or operating conditions, but also across different machines. This illustrates the promising potential of the proposed approach for cross-machine fault diagnosis tasks in real-world applications.Comment: 7 pages, 1 figure, 5 tables, submitted to IEEE PESGRE 202

Active Foundational Models for Fault Diagnosis of Electrical Motors

Fault detection and diagnosis of electrical motors are of utmost importance in ensuring the safe and reliable operation of several industrial systems. Detection and diagnosis of faults at the incipient stage allows corrective actions to be taken in order to reduce the severity of faults. The existing data-driven deep learning approaches for machine fault diagnosis rely extensively on huge amounts of labeled samples, where annotations are expensive and time-consuming. However, a major portion of unlabeled condition monitoring data is not exploited in the training process. To overcome this limitation, we propose a foundational model-based Active Learning framework that utilizes less amount of labeled samples, which are most informative and harnesses a large amount of available unlabeled data by effectively combining Active Learning and Contrastive Self-Supervised Learning techniques. It consists of a transformer network-based backbone model trained using an advanced nearest-neighbor contrastive self-supervised learning method. This approach empowers the backbone to learn improved representations of samples derived from raw, unlabeled vibration data. Subsequently, the backbone can undergo fine-tuning to address a range of downstream tasks, both within the same machines and across different machines. The effectiveness of the proposed methodology has been assessed through the fine-tuning of the backbone for multiple target tasks using three distinct machine-bearing fault datasets. The experimental evaluation demonstrates a superior performance as compared to existing state-of-the-art fault diagnosis methods with less amount of labeled data.Comment: 30 pages, 2 figures, 7 table

BioBridge: Bridging Biomedical Foundation Models via Knowledge Graphs

Foundation models (FMs) are able to leverage large volumes of unlabeled data to demonstrate superior performance across a wide range of tasks. However, FMs developed for biomedical domains have largely remained unimodal, i.e., independently trained and used for tasks on protein sequences alone, small molecule structures alone, or clinical data alone. To overcome this limitation of biomedical FMs, we present BioBridge, a novel parameter-efficient learning framework, to bridge independently trained unimodal FMs to establish multimodal behavior. BioBridge achieves it by utilizing Knowledge Graphs (KG) to learn transformations between one unimodal FM and another without fine-tuning any underlying unimodal FMs. Our empirical results demonstrate that BioBridge can beat the best baseline KG embedding methods (on average by around 76.3%) in cross-modal retrieval tasks. We also identify BioBridge demonstrates out-of-domain generalization ability by extrapolating to unseen modalities or relations. Additionally, we also show that BioBridge presents itself as a general purpose retriever that can aid biomedical multimodal question answering as well as enhance the guided generation of novel drugs

HYTREL: Hypergraph-enhanced Tabular Data Representation Learning

Language models pretrained on large collections of tabular data have demonstrated their effectiveness in several downstream tasks. However, many of these models do not take into account the row/column permutation invariances, hierarchical structure, etc. that exist in tabular data. To alleviate these limitations, we propose HYTREL, a tabular language model, that captures the permutation invariances and three more structural properties of tabular data by using hypergraphs - where the table cells make up the nodes and the cells occurring jointly together in each row, column, and the entire table are used to form three different types of hyperedges. We show that HYTREL is maximally invariant under certain conditions for tabular data, i.e., two tables obtain the same representations via HYTREL iff the two tables are identical up to permutations. Our empirical results demonstrate that HYTREL consistently outperforms other competitive baselines on four downstream tasks with minimal pretraining, illustrating the advantages of incorporating the inductive biases associated with tabular data into the representations. Finally, our qualitative analyses showcase that HYTREL can assimilate the table structures to generate robust representations for the cells, rows, columns, and the entire table.Comment: NeurIPS 2023 (spotlight

Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selection

Large Language Models (LLMs) can adapt to new tasks via in-context learning (ICL). ICL is efficient as it does not require any parameter updates to the trained LLM, but only few annotated examples as input for the LLM. In this work, we investigate an active learning approach for ICL, where there is a limited budget for annotating examples. We propose a model-adaptive optimization-free algorithm, termed AdaICL, which identifies examples that the model is uncertain about, and performs semantic diversity-based example selection. Diversity-based sampling improves overall effectiveness, while uncertainty sampling improves budget efficiency and helps the LLM learn new information. Moreover, AdaICL poses its sampling strategy as a Maximum Coverage problem, that dynamically adapts based on the model's feedback and can be approximately solved via greedy algorithms. Extensive experiments on nine datasets and seven LLMs show that AdaICL improves performance by 4.4% accuracy points over SOTA (7.7% relative improvement), is up to 3x more budget-efficient than performing annotations uniformly at random, while it outperforms SOTA with 2x fewer ICL examples

Isolation and characterization of Newcastle disease virus from vaccinated commercial layer chicken

Aim: Newcastle disease (ND) is an infectious, highly contagious and destructive viral disease of poultry and controlled by vaccination. In spite of vaccination, incidence of ND was reported in commercial layers with gastrointestinal lesions. This study was undertaken to assess the prevalence and pathotypes of Newcastle disease virus (NDV) involved in gastrointestinal tract abnormalities of vaccinated commercial layer chicken of Namakkal region for a period of three years from 2008 and 2011. Materials and Methods: Pooled tissue (trachea, lung, spleen, proventriculus, intestine and caecal tonsils) samples collected from dead birds on postmortem examination from 100 layer flocks above 20 weeks of age with gastrointestinal lesions were subjected to isolation of NDV in embryonated specific pathogen free (SPF) chicken eggs. Mean death time (MDT) and intracerebral pathogenicity index of the isolates were characterized. Flock details were collected from NDV positive flocks to assess the prevalence and impact of NDV on vaccinated commercial layer chicken. Results: Among the 100 flocks examined Newcastle disease virus was detected in 14 flocks as a single infection and 10 flocks as combined infections with worm infestation, necrotic enteritis and coccidiosis. Chicken embryo mean death time (MDT) and intracerebral pathogenicity index (ICPI) values ranged from 50.4 to 96.0 hrs and from 0.650 to 1.675 respectively. Affected birds showed anorexia, diarrohea and drop in egg production. Macropathologically, matting of vent feathers, petechial haemorrhage on the tip of proventricular papilla, caecal tonsils and degeneration of ovarian follicles were noticed. The incidence of ND was most commonly noticed in 20-50 wk of age and between the months of September to November. Morbidity rate varied from 5% to 10% in the NDV alone affected flocks and 5 to 15% in NDV with other concurrent infections. Egg production drop from the expected level ranged between 3 to 7 % in ND and 5 to 10 % in concurrent infections. Average mortality in NDV and concurrently affected (NDV and Coccidiosis) flocks were 2.89% and 3.50 % respectively. Conclusion: The present study revealed 24 % of gastrointestinal tract abnormalities in commercial layer chicken were caused by various pathotypes of Newcastle disease virus. The virus caused the disease as single and concurrently with other diseases. Vaccination minimized the clinical manifestation and lesions even in velogenic virus affected flocks