Transformer Encoder with Multiscale Deep Learning for Pain Classification Using Physiological Signals

Pain is a serious worldwide health problem that affects a vast proportion of the population. For efficient pain management and treatment, accurate classification and evaluation of pain severity are necessary. However, this can be challenging as pain is a subjective sensation-driven experience. Traditional techniques for measuring pain intensity, e.g. self-report scales, are susceptible to bias and unreliable in some instances. Consequently, there is a need for more objective and automatic pain intensity assessment strategies. In this paper, we develop PainAttnNet (PAN), a novel transfomer-encoder deep-learning framework for classifying pain intensities with physiological signals as input. The proposed approach is comprised of three feature extraction architectures: multiscale convolutional networks (MSCN), a squeeze-and-excitation residual network (SEResNet), and a transformer encoder block. On the basis of pain stimuli, MSCN extracts short- and long-window information as well as sequential features. SEResNet highlights relevant extracted features by mapping the interdependencies among features. The third module employs a transformer encoder consisting of three temporal convolutional networks (TCN) with three multi-head attention (MHA) layers to extract temporal dependencies from the features. Using the publicly available BioVid pain dataset, we test the proposed PainAttnNet model and demonstrate that our outcomes outperform state-of-the-art models. These results confirm that our approach can be utilized for automated classification of pain intensity using physiological signals to improve pain management and treatment

A Unique Methodology For Implementing High School Capstone Experiences Through Teacher Professional Development

Innovators and abstract thinkers - students who question why are going to be the future of engineering, of science and cures for diseases. Rarely do students ask where and how innovation is created. Students, particularly post-secondary students have lost their curiosity and they have lost their ability to question. Why? Because the relationship between theory and application has been removed from our high schools. Although the term “STEM” is generally used, students do not appear to understand the importance of core STEM principles such as Newton’s 2nd law and therefore do not understand the influence these basic algorithms have in daily life. In recent decades, high school education has focused on quizzes and exams, state and national standardize testing and SATs. More emphasis is placed on performing well on these exams, focusing on memorization and test taking rather than on thorough comprehension. The question is, “how do you translate theory to application in the high school classroom?” Students’ knowledge and engagement are only as good as their teachers. Educators need to be given the proper tools, resources, and knowledge. CAPSULE, a capstone-based experience provides tools, resources, and knowledge to enhance the teaching and learning involvement. CAPSULE teaches and promotes inquiry, exploration and application rather than just theory. The methodology engages and educates hands-on learning, teamwork and multiple solutions through the engineering design process (EDP). The theory behind innovation is the motivation for CAPSULE – to teach and engage teachers using 3D modeling, EDP, and project-based learning to create a high school capstone experience. This paper presents a new approach of teaching STEM related courses to high school students. The methodology presented is on “training the trainer” to enable and empower teachers to master and utilize this new approach.

Uncertainty Quantification in Neural-Network Based Pain Intensity Estimation

Improper pain management can lead to severe physical or mental consequences, including suffering, and an increased risk of opioid dependency. Assessing the presence and severity of pain is imperative to prevent such outcomes and determine the appropriate intervention. However, the evaluation of pain intensity is challenging because different individuals experience pain differently. To overcome this, researchers have employed machine learning models to evaluate pain intensity objectively. However, these efforts have primarily focused on point estimation of pain, disregarding the inherent uncertainty and variability present in the data and model. Consequently, the point estimates provide only partial information for clinical decision-making. This study presents a neural network-based method for objective pain interval estimation, incorporating uncertainty quantification. This work explores three algorithms: the bootstrap method, lower and upper bound estimation (LossL) optimized by genetic algorithm, and modified lower and upper bound estimation (LossS) optimized by gradient descent algorithm. Our empirical results reveal that LossS outperforms the other two by providing a narrower prediction interval. As LossS outperforms, we assessed its performance in three different scenarios for pain assessment: (1) a generalized approach (single model for the entire population), (2) a personalized approach (separate model for each individual), and (3) a hybrid approach (separate model for each cluster of individuals). Our findings demonstrate the hybrid approach's superior performance, with notable practicality in clinical contexts. It has the potential to be a valuable tool for clinicians, enabling objective pain intensity assessment while taking uncertainty into account. This capability is crucial in facilitating effective pain management and reducing the risks associated with improper treatment.Comment: 26 pages, 5 figures, 9 table

Review and Analysis of Pain Research Literature through Keyword Co-occurrence Networks

Pain is a significant public health problem as the number of individuals with a history of pain globally keeps growing. In response, many synergistic research areas have been coming together to address pain-related issues. This work conducts a review and analysis of a vast body of pain-related literature using the keyword co-occurrence network (KCN) methodology. In this method, a set of KCNs is constructed by treating keywords as nodes and the co-occurrence of keywords as links between the nodes. Since keywords represent the knowledge components of research articles, analysis of KCNs will reveal the knowledge structure and research trends in the literature. This study extracted and analyzed keywords from 264,560 pain-related research articles indexed in IEEE, PubMed, Engineering Village, and Web of Science published between 2002 and 2021. We observed rapid growth in pain literature in the last two decades: the number of articles has grown nearly threefold, and the number of keywords has grown by a factor of 7. We identified emerging and declining research trends in sensors/methods, biomedical, and treatment tracks. We also extracted the most frequently co-occurring keyword pairs and clusters to help researchers recognize the synergies among different pain-related topics

Principles for Designing Green, Lean, and Smart Microfactories: Chicken as a Model

Industrial revolutions have gone through four phases: steam, electricity, electronics, and Industry 4.0. Through all these four industrial revolutions, efficiency, productivity, quality, and automation have been greatly improved. However, the manufacturing processes created by humans have had disastrous consequences on the environment leading to a gigantic “climate change” problem. To mitigate climate change, engineers, and manufacturers all over the world have stepped up the research into cradle-to-cradle designs and sustainable manufacturing practices inspired by the designs and value cycles in nature. Bio-inspired designs have been gaining momentum to create products and manufacturing methods that are eco-friendly. All manufacturing (of a fruit, an organism such as a human baby) in nature happens in microfactories such as a womb, a leaf, a flower, or a chicken oviduct whose products are eggs. The product (egg) and the manufacturing process (chicken oviduct) are both green (eco-effective), lean (built with minimal resources), and smart (sensors and Internet of Things). Using a chicken as a model, this book chapter presents a set of metrics for green, lean, and smart attributes, which engineers can use to design products and microfactories

pyKCN: A Python Tool for Bridging Scientific Knowledge

The study of research trends is pivotal for understanding scientific development on specific topics. Traditionally, this involves keyword analysis within scholarly literature, yet comprehensive tools for such analysis are scarce, especially those capable of parsing large datasets with precision. pyKCN, a Python toolkit, addresses this gap by automating keyword cleaning, extraction and trend analysis from extensive academic corpora. It is equipped with modules for text processing, deduplication, extraction, and advanced keyword co-occurrence and analysis, providing a granular view of research trends. This toolkit stands out by enabling researchers to visualize keyword relationships, thereby identifying seminal works and emerging trends. Its application spans diverse domains, enhancing scholars' capacity to understand developments within their fields. The implications of using pyKCN are significant. It offers an empirical basis for predicting research trends, which can inform funding directions, policy-making, and academic curricula. The code source and details can be found on: https://github.com/zhenyuanlu/pyKC

DETC 2008-49323 EFFECT OF BASE MATERIAL ON RFID TAG READABILITY

ABSTRACT RFID tags are commonly used to track products in many applications including retail and pharmaceutical industries. Tag readability is detrimental to the successful implementation and use of RFID in commercial applications. Tag readability is adversely affected when tags are adhered to metal surfaces such as metal cans and containers. This paper studies the effect of base material on tag readability. We have designed and built an experimental setup in the RFID lab to perform our studies. Our studies show that the metal effect is influenced by the distance between the tag and the base material (padding material) and the distance between the tag and its RFID reader. The studies also reveal the threshold values for improving tag readability

Personalized Deep Bi-LSTM RNN Based Model for Pain Intensity Classification Using EDA Signal

Automatic pain intensity assessment from physiological signals has become an appealing approach, but it remains a largely unexplored research topic. Most studies have used machine learning approaches built on carefully designed features based on the domain knowledge available in the literature on the time series of physiological signals. However, a deep learning framework can automate the feature engineering step, enabling the model to directly deal with the raw input signals for real-time pain monitoring. We investigated a personalized Bidirectional Long short-term memory Recurrent Neural Networks (BiLSTM RNN), and an ensemble of BiLSTM RNN and Extreme Gradient Boosting Decision Trees (XGB) for four-category pain intensity classification. We recorded Electrodermal Activity (EDA) signals from 29 subjects during the cold pressor test. We decomposed EDA signals into tonic and phasic components and augmented them to original signals. The BiLSTM-XGB model outperformed the BiLSTM classification performance and achieved an average F1-score of 0.81 and an Area Under the Receiver Operating Characteristic curve (AUROC) of 0.93 over four pain states: no pain, low pain, medium pain, and high pain. We also explored a concatenation of the deep-learning feature representations and a set of fourteen knowledge-based features extracted from EDA signals. The XGB model trained on this fused feature set showed better performance than when it was trained on component feature sets individually. This study showed that deep learning could let us go beyond expert knowledge and benefit from the generated deep representations of physiological signals for pain assessment