Robust Intent Classification using Bayesian LSTM for Clinical Conversational Agents (CAs)

Conversational Agents (CAs) are software programs that replicate hu-man conversations using machine learning (ML) and natural language processing (NLP). CAs are currently being utilised for diverse clinical applications such as symptom checking, health monitoring, medical triage and diagnosis. Intent clas-sification (IC) is an essential task of understanding user utterance in CAs which makes use of modern deep learning (DL) methods. Because of the inherent model uncertainty associated with those methods, accuracy alone cannot be relied upon in clinical applications where certain errors may compromise patient safety. In this work, we employ Bayesian Long Short-Term Memory Networks (LSTMs) to calculate model uncertainty for IC, with a specific emphasis on symptom checker CAs. This method provides a certainty measure with IC prediction that can be utilised in assuring safe response from CAs. We evaluated our method on in-distribution (ID) and out-of-distribution (OOD) data and found mean uncer-tainty to be much higher for OOD data. These findings suggest that our method is robust to OOD utterances and can detect non-understanding errors in CAs

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Context: Machine Learning (ML) has been at the heart of many innovations over the past years. However, including it in so-called 'safety-critical' systems such as automotive or aeronautic has proven to be very challenging, since the shift in paradigm that ML brings completely changes traditional certification approaches. Objective: This paper aims to elucidate challenges related to the certification of ML-based safety-critical systems, as well as the solutions that are proposed in the literature to tackle them, answering the question 'How to Certify Machine Learning Based Safety-critical Systems?'. Method: We conduct a Systematic Literature Review (SLR) of research papers published between 2015 to 2020, covering topics related to the certification of ML systems. In total, we identified 217 papers covering topics considered to be the main pillars of ML certification: Robustness, Uncertainty, Explainability, Verification, Safe Reinforcement Learning, and Direct Certification. We analyzed the main trends and problems of each sub-field and provided summaries of the papers extracted. Results: The SLR results highlighted the enthusiasm of the community for this subject, as well as the lack of diversity in terms of datasets and type of models. It also emphasized the need to further develop connections between academia and industries to deepen the domain study. Finally, it also illustrated the necessity to build connections between the above mention main pillars that are for now mainly studied separately. Conclusion: We highlighted current efforts deployed to enable the certification of ML based software systems, and discuss some future research directions.Comment: 60 pages (92 pages with references and complements), submitted to a journal (Automated Software Engineering). Changes: Emphasizing difference traditional software engineering / ML approach. Adding Related Works, Threats to Validity and Complementary Materials. Adding a table listing papers reference for each section/subsection

Transverse mechanical responses of unidirectional fibre-reinforced polymers:DEM simulation and deep learning prediction

Fibre-reinforced polymer (FRP) composites have broad applications in automotive, aerospace, construction, and marine sectors due to their excellent physical and mechanical properties, such as lightweight, high strength, high stiffness, and corrosion resistance. Laminated FRP composites are among the most commonly used FRP composites that consist of several unidirectional (UD) FRP laminae, which were also widely used. However, predicting mechanical properties and damage progression of UD FRP composites at the laminae scale with high accuracy is still a challenging problem as the damage evolution and failure mechanism is obviously far more complex than those of monolithic materials, and a precise approach is usually time-consuming. In this thesis, firstly, a 2D plane strain computational micromechanics Discrete Element Method (DEM) model is used to investigate the mechanical behaviour of a UD FRP composite lamina subjected to transverse loading. The damage progression and stress-strain response of the laminae are analysed. The DEM simulation results agree with those from the FEM and experimental tests, which can offer accurate and reliable results for machine learning predictions. Thus, a deep neural network (DNN) model for predicting the macroscopic transverse mechanical properties of UD FRP lamina is investigated when considering the different fibre radii and ratios. The DNN model has a strong ability to predict both the transverse tensile strength and Young’s modulus of the UD FRP composite lamina accurately. Then the predictive models for predicting the microscopic cracks are constructed. Two simple ML models are first generated based on the data extracted from 500 DEM model simulations. However, the prediction results indicate that the relation to the target problem is complicated. The deep learning (DL) model is therefore developed to explore the complex inner relation of the prediction problem. Based on the results of 1600 DEM simulations, a multi-layer DNN model with back-propagation is constructed for the predictions of the crack areas of an FRP composite lamina. It can be found that most of the initial cracks and the overall trends of cracking paths can be predicted successfully. Then, a DNN model is applied to predict the initial and second crack in order in the FRP laminae to obtain a fast determination of the crack initiation, which results could demonstrate the feasibility and effectiveness of the method. In addition to the defect-free UD FRP lamina, the transverse mechanical response of defective lamina is studied using the DEM model. The DEM model is recalibrated for analysing the effects of the presence of defects in the RVE of the lamina. The crack initiation and propagation in defective RVEs with different fibre distributions are analysed and compared. In addition, the effects of the radii of matrix defects and the distribution of defects on stress-strain responses are also investigated, which shows excellent capabilities in predicting mechanical behaviour. It is found that the crack initiation is highly dependent on the defects. Therefore, back-propagation deep neural network (DNN) models are developed to quickly determine crack initiation and instantaneous critical load of the RVEs based on the data generated by 1000 DEM simulations. The results show that the initial crack and the critical stress of the laminae can be accurately and efficiently predicted by the data-driven DNN models with full consideration of randomly distributed defects