Multimodal Convolutional Neural Networks to Detect Fetal Compromise During Labor and Delivery

The gold standard to assess whether a baby is at risk of oxygen deprivation during childbirth, is monitoring continuously the fetal heart rate with cardiotocography (CTG). The aim is to identify babies that could benefit from an emergency operative delivery (e.g., Cesarean section), in order to prevent death or permanent brain injury. The long, dynamic and complex CTG patterns are poorly understood and known to have high false positive and false negative rates. Visual interpretation by clinicians is challenging and reliable accurate fetal monitoring in labor remains an enormous unmet medical need. In this work, we applied deep learning methods to achieve data-driven automated CTG evaluation. Multimodal Convolutional Neural Network (MCNN) and Stacked MCNN models were used to analyze the largest available database of routinely collected CTG and linked clinical data (comprising more than 35000 births). We also assessed in detail the impact of the signal quality on the MCNN performance. On a large hold-out testing set from Oxford (n= 4429 births), MCNN improved the prediction of cord acidemia at birth when compared with Clinical Practice and previous computerized approaches. On two external datasets, MCNN demonstrated better performance compared to current feature extraction-based methods. Our group is the first to apply deep learning for the analysis of CTG. We conclude that MCNN hold potential for the prediction of cord acidemia at birth and further work is warranted. Despite the advances, our deep learning models are currently not suitable for the detection of severe fetal injury in the absence of cord acidemia - a heterogeneous, small, and poorly understood group. We suggest that the most promising way forward are hybrid approaches to CTG interpretation in labor, in which different diagnostic models can estimate the risk for different types of fetal compromise, incorporating clinical knowledge with data-driven analyses

Prediction of fetal blood pressure during labour with deep learning techniques

Our objective is to develop a model for the prediction of minimum fetal blood pressure (FBP) during fetal heart rate (FHR) decelerations. Experimental data from umbilical occlusions in near-term fetal sheep (2698 occlusions from 57 near-term lambs) were used to train a convolutional neural network. This model was then used to estimate FBP for decelerations extracted from the final 90 min of 53,445 human FHR signals collected using cardiotocography. Minimum sheep FBP was predicted with a mean absolute error of 6.7 mmHg (25th, 50th, 75th percentiles of 2.3, 5.2, 9.7 mmHg), mean absolute percentage errors of 17.3% (5.5%, 12.5%, 23.9%) and a coefficient of determination 2=0.36. While the model was unable to clearly predict severe compromise at birth in humans, there is positive evidence that such a model could predict human FBP with further development. The neural network is capable of predicting FBP for many of the sheep decelerations accurately but performed far from satisfactory at identifying FHR segments that correspond to the highest or lowest minimum FBP. These results indicate that with further work and a larger, more variable training dataset, the model could achieve higher accuracy

Machine learning on cardiotocography data to classify fetal outcomes: A scoping review

Introduction: Uterine contractions during labour constrict maternal blood flow and oxygen delivery to the developing baby, causing transient hypoxia. While most babies are physiologically adapted to withstand such intrapartum hypoxia, those exposed to severe hypoxia or with poor physiological reserves may experience neurological injury or death during labour. Cardiotocography (CTG) monitoring was developed to identify babies at risk of hypoxia by detecting changes in fetal heart rate (FHR) patterns. CTG monitoring is in widespread use in intrapartum care for the detection of fetal hypoxia, but the clinical utility is limited by a relatively poor positive predictive value (PPV) of an abnormal CTG and significant inter and intra observer variability in CTG interpretation. Clinical risk and human factors may impact the quality of CTG interpretation. Misclassification of CTG traces may lead to both under-treatment (with the risk of fetal injury or death) or over-treatment (which may include unnecessary operative interventions that put both mother and baby at risk of complications). Machine learning (ML) has been applied to this problem since early 2000 and has shown potential to predict fetal hypoxia more accurately than visual interpretation of CTG alone. To consider how these tools might be translated for clinical practice, we conducted a review of ML techniques already applied to CTG classification and identified research gaps requiring investigation in order to progress towards clinical implementation. Materials and method: We used identified keywords to search databases for relevant publications on PubMed, EMBASE and IEEE Xplore. We used Preferred Reporting Items for Systematic Review and Meta-Analysis for Scoping Reviews (PRISMA-ScR). Title, abstract and full text were screened according to the inclusion criteria. Results: We included 36 studies that used signal processing and ML techniques to classify CTG. Most studies used an open-access CTG database and predominantly used fetal metabolic acidosis as the benchmark for hypoxia with varying pH levels. Various methods were used to process and extract CTG signals and several ML algorithms were used to classify CTG. We identified significant concerns over the practicality of using varying pH levels as the CTG classification benchmark. Furthermore, studies needed to be more generalised as most used the same database with a low number of subjects for an ML study. Conclusion: ML studies demonstrate potential in predicting fetal hypoxia from CTG. However, more diverse datasets, standardisation of hypoxia benchmarks and enhancement of algorithms and features are needed for future clinical implementation.</p

Machine learning algorithms combining slope deceleration and fetal heart rate features to predict acidemia

Electronic fetal monitoring (EFM) is widely used in intrapartum care as the standard method for monitoring fetal well-being. Our objective was to employ machine learning algorithms to predict acidemia by analyzing specific features extracted from the fetal heart signal within a 30 min window, with a focus on the last deceleration occurring closest to delivery. To achieve this, we conducted a case–control study involving 502 infants born at Miguel Servet University Hospital in Spain, maintaining a 1:1 ratio between cases and controls. Neonatal acidemia was defined as a pH level below 7.10 in the umbilical arterial blood. We constructed logistic regression, classification trees, random forest, and neural network models by combining EFM features to predict acidemia. Model validation included assessments of discrimination, calibration, and clinical utility. Our findings revealed that the random forest model achieved the highest area under the receiver characteristic curve (AUC) of 0.971, but logistic regression had the best specificity, 0.879, for a sensitivity of 0.95. In terms of clinical utility, implementing a cutoff point of 31% in the logistic regression model would prevent unnecessary cesarean sections in 51% of cases while missing only 5% of acidotic cases. By combining the extracted variables from EFM recordings, we provide a practical tool to assist in avoiding unnecessary cesarean sections

Cardiotocography Signal Abnormality Detection based on Deep Unsupervised Models

Cardiotocography (CTG) is a key element when it comes to monitoring fetal well-being. Obstetricians use it to observe the fetal heart rate (FHR) and the uterine contraction (UC). The goal is to determine how the fetus reacts to the contraction and whether it is receiving adequate oxygen. If a problem occurs, the physician can then respond with an intervention. Unfortunately, the interpretation of CTGs is highly subjective and there is a low inter- and intra-observer agreement rate among practitioners. This can lead to unnecessary medical intervention that represents a risk for both the mother and the fetus. Recently, computer-assisted diagnosis techniques, especially based on artificial intelligence models (mostly supervised), have been proposed in the literature. But, many of these models lack generalization to unseen/test data samples due to overfitting. Moreover, the unsupervised models were applied to a very small portion of the CTG samples where the normal and abnormal classes are highly separable. In this work, deep unsupervised learning approaches, trained in a semi-supervised manner, are proposed for anomaly detection in CTG signals. The GANomaly framework, modified to capture the underlying distribution of data samples, is used as our main model and is applied to the CTU-UHB dataset. Unlike the recent studies, all the CTG data samples, without any specific preferences, are used in our work. The experimental results show that our modified GANomaly model outperforms state-of-the-arts. This study admit the superiority of the deep unsupervised models over the supervised ones in CTG abnormality detection

Challenges of developing robust AI for intrapartum fetal heart rate monitoring

Background: CTG remains the only non-invasive tool available to the maternity team for continuous monitoring of fetal well-being during labour. Despite widespread use and investment in staff training, difficulty with CTG interpretation continues to be identified as a problem in cases of fetal hypoxia, which often results in permanent brain injury. Given the recent advances in AI, it is hoped that its application to CTG will offer a better, less subjective and more reliable method of CTG interpretation. Objectives: This mini-review examines the literature and discusses the impediments to the success of AI application to CTG thus far. Prior randomised control trials (RCTs) of CTG decision support systems are reviewed from technical and clinical perspectives. A selection of novel engineering approaches, not yet validated in RCTs, are also reviewed. The review presents the key challenges that need to be addressed in order to develop a robust AI tool to identify fetal distress in a timely manner so that appropriate intervention can be made. Results: The decision support systems used in three RCTs were reviewed, summarising the algorithms, the outcomes of the trials and the limitations. Preliminary work suggests that the inclusion of clinical data can improve the performance of AI-assisted CTG. Combined with newer approaches to the classification of traces, this offers promise for rewarding future development

Multilingual Text Detection on Scene Images using MASK RCNN Method

Στη νέα εποχή της τεχνολογίας, όπου οι καινοτομίες αυξάνονται μέρα με τη μέρα, οι νέες ιδέες, μέθοδοι, διαδικασίες στον τομέα της επιστήμης υπολογιστών απαιτούνται όλο και περισσότερο. Ένα από τα πιο ενεργά θέματα αυτές τις μέρες είναι η «ανίχνευση και αναγνώριση κειμένου» σε εικόνες ή βίντεο. Οι ακριβείς πληροφορίες που υπάρχουν στην εικόνα είναι πολύ χρήσιμες για ένα ευρύ φάσμα εφαρμογών στην πραγματική ζωή. Ωστόσο, είναι μια πολύ περίπλοκη διαδικασία να εντοπιστεί και να αναγνωριστεί κείμενο σε εικόνες σκηνής. Η ανίχνευση και αναγνώριση κειμένου σε εικόνες σκηνής, λόγω της ποικιλίας των εφαρμογών που υπάρχουν στην αγορά, αναζητά την προσοχή της κοινότητας της τεχνολογίας των υπολογιστών όλο και περισσότερο. Υπάρχουν ορισμένα προβλήματα σε αυτόν τον τομέα που δεν έχουν επιλυθεί ακόμα όπως είναι πολυγλωσσία, χρώματα, προσανατολισμοί, γραμματοσειρές, στυλ. Οι πρόσφατες εξελίξεις στη βαθιά μάθηση έχουν αυξήσει την προσοχή δυνητικών ερευνητών στην ανίχνευση κειμένου. Η αποτελεσματικότητα των Convolutional Neural Networks βασίζεται σε μεγάλο βαθμό στην απόδοση του αλγορίθμου που υιοθετείται για την ανίχνευση αντικειμένου. Υπάρχουν πολλές δυσκολίες που πρέπει να αντιμετωπιστούν οι οποίες σχετίζονται με την ανίχνευση κειμένου σκηνής. Το μεγαλύτερο πρόβλημα είναι ότι οι περισσότερες από τις μεθόδους που χρησιμοποιούνται για την ανίχνευση κειμένου δείχνουν καλύτερη απόδοση όταν οι συνθήκες είναι υπό έλεγχο, όταν οι περιπτώσεις στις οποίες το κείμενο έχει κανονικό σχήμα και κανονική αναλογία. Λόγω περιορισμένων μορφών αναπαράστασης κειμένου και περιορισμένου δεκτικού μεγέθους CNN, οι μέθοδοι αυτοί δεν εντοπίζουν τις πολύπλοκες σκηνές, όπως κείμενα που έχουν αυθαίρετο σχήμα ή είναι μακρά κείμενα. Η προτεινόμενη μέθοδος που εφαρμόζεται και δοκιμάζεται είναι η Mask RCNN για να προσφέρει ένα πλαίσιο που έχει μια ισχυρή βάση και προσφέρει πολλά πλεονεκτήματα θεσμικής σαφήνειας στην έννοια, την ευελιξία, την ευρωστία και τον γρήγορο χρόνο εκμάθησης.In the new era of technology, where innovations come day by day the new ideas, methods, procedures in the field of Computer sciences getting more advance. One of them research becomes most active topic these days is ‘text detection and recognition’ presents in images or videos. The accurate information present in the text is very useful for a wide range of real-life applications. However, it is a very complicated assignment to localize and read texts from natural scene images. Scene text detection and recognition application, due to majority and variety of these applications present in the market, it seeks the attention of community to the computer technology more and become curios. There are some problems which are unsolved till know in the world of text detection which are languages, colors, orientation, fonts, style that need to be resolved. The recent advancements in deep learning have increased the attention of potential researchers towards scene text detection. CNN is designed in the way that it automatically adapts spatial hierarchies of Text detection Processing features through using multiple building blocks - layers. First, it collects the text each word separately and after detecting the different parts of text it recollects the whole image and then present an output. In this report it is analyzed the techniques like LOMO and PMTD etc. for text detection. Our proposed method is using MASK RCNN technique and it is implemented and tested in order to offer a framework that has a powerful baseline and offers so many advantages such as flexibility, robustness, fast time of training and inference. All the methods help us to achieve two different activities which are instance segmentation and text detection on scene images. The methods that are proposed in the research have already offers some application that include the Multilingual text detection on scene images, but this is not meant that these applications are ideal to use or perfect. There is a lag and missing features in these applications and have a room for improvement which can help in achieving better results in terms of boundary detection. The world of technology improving day by day and being update with the technology and contribute in the technology is the best way to express greeting or thanks to the technolog

2022 - The Third Annual Fall Symposium of Student Scholars

The full program book from the Fall 2022 Symposium of Student Scholars, held on November 17, 2022. Includes abstracts from the presentations and posters.https://digitalcommons.kennesaw.edu/sssprograms/1026/thumbnail.jp

Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challenges

Artificial General Intelligence (AGI), possessing the capacity to comprehend, learn, and execute tasks with human cognitive abilities, engenders significant anticipation and intrigue across scientific, commercial, and societal arenas. This fascination extends particularly to the Internet of Things (IoT), a landscape characterized by the interconnection of countless devices, sensors, and systems, collectively gathering and sharing data to enable intelligent decision-making and automation. This research embarks on an exploration of the opportunities and challenges towards achieving AGI in the context of the IoT. Specifically, it starts by outlining the fundamental principles of IoT and the critical role of Artificial Intelligence (AI) in IoT systems. Subsequently, it delves into AGI fundamentals, culminating in the formulation of a conceptual framework for AGI's seamless integration within IoT. The application spectrum for AGI-infused IoT is broad, encompassing domains ranging from smart grids, residential environments, manufacturing, and transportation to environmental monitoring, agriculture, healthcare, and education. However, adapting AGI to resource-constrained IoT settings necessitates dedicated research efforts. Furthermore, the paper addresses constraints imposed by limited computing resources, intricacies associated with large-scale IoT communication, as well as the critical concerns pertaining to security and privacy

Multimodal Analytics for Healthcare

The ailing healthcare system demands effective autonomous solutions to improve service and provide individualize care, while reducing the burden on the scarce healthcare workforce. Most of these solutions require a multidisciplinary approach that combines healthcare with computational abilities. The work presented in this thesis introduces a multimodal multiview network along with methods and solutions that leverage inexpensive visual sensors and computers to monitor healthcare. One of the most prominent outcomes of this work includes enabling the medical analysis of ICU conditions such as sleep disorders, decubitus ulcerations, and hospital acquired infections, which are preventable and negatively affect patients' health population. The problems tackled include patient pose classification, pose motion analysis and summarization, role representation and identification, and activity and event logging in natural hospital settings. These problems are addressed via a non-intrusive non-disruptive multimodal multiview sensor network (Medical Internet-of-Things). The multimodal data is combined with coupled-optimization to estimate source weights and accurately classify patient poses. Pose patterns such as pose transitions are represented using deep convolutional features and pose duration is modelled via segments. The proposed techniques serve to differentiate between poses and pseudo-poses (transitory poses) and create effective motion summaries. The role representation is tackled using novel appearance and semantic interaction maps to assign generic labels to individuals (doctor, nurse, visitor, etc) without using identifiable information (e.g., facetracking or badges), which is prohibited in healthcare applications. Finally, activity and event analysis is tackled using a new contextual aspect frames where aspect bases and weights are learned and then used to reconstruct activities. The objective of this thesis is to enable the development, evaluation, and optimization of individualized therapies, standards-of-care, room infrastructural designs, and clinical workflows and procedures