Challenges and Opportunities of Using Transformer-Based Multi-Task Learning in NLP Through ML Lifecycle: A Survey

The increasing adoption of natural language processing (NLP) models across industries has led to practitioners' need for machine learning systems to handle these models efficiently, from training to serving them in production. However, training, deploying, and updating multiple models can be complex, costly, and time-consuming, mainly when using transformer-based pre-trained language models. Multi-Task Learning (MTL) has emerged as a promising approach to improve efficiency and performance through joint training, rather than training separate models. Motivated by this, we first provide an overview of transformer-based MTL approaches in NLP. Then, we discuss the challenges and opportunities of using MTL approaches throughout typical ML lifecycle phases, specifically focusing on the challenges related to data engineering, model development, deployment, and monitoring phases. This survey focuses on transformer-based MTL architectures and, to the best of our knowledge, is novel in that it systematically analyses how transformer-based MTL in NLP fits into ML lifecycle phases. Furthermore, we motivate research on the connection between MTL and continual learning (CL), as this area remains unexplored. We believe it would be practical to have a model that can handle both MTL and CL, as this would make it easier to periodically re-train the model, update it due to distribution shifts, and add new capabilities to meet real-world requirements

Comparison of Multidetector-Row Computed Tomography and Duplex Doppler Ultrasonography in Detecting Atherosclerotic Carotid Plaques Complicated with Intraplaque Hemorrhage

This study compared sensitivity and specificity of multidetector-row computed tomography and duplex Doppler ultrasonography in detecting atherosclerotic carotid plaques complicated with intraplaque hemorrhage. Carotid plaques from 50 patients operated for carotid artery stenosis were analyzed. Carotid endarterectomy was performed within one week of diagnostic evaluation. Results of multidetector-row computed tomography and duplex Doppler ultrasonography diagnostic evaluation were compared with results of histological analysis of the same plaque areas. American Heart Association classification of atherosclerotic plaques was applied for histological classification. Median tissue density of carotid plaques complicated with intraplaque hemorrhage was 14.7 Hounsfield units. Median tissue density of noncalcified segments of uncomplicated plaques was 54.3 Hounsfield units (p=0.00003). The highest tissue density observed for complicated plaques was 31.8 Hounsfield units. Multidetector-row computed tomography detected plaques complicated with hemorrhage with sensitivity of 100% and specificity of 70.4%, with tissue density of 33.8 Hounsfield units as a threshold value. Duplex Doppler ultrasonography plaque analysis based on visual in-line classification showed sensitivity of 21.7% and specificity of 89.6% in detecting plaques complicated with intraplaque hemorrhage. Multidetector-row computed tomography showed a very high level of sensitivity and a moderate level of specificity in detecting atherosclerotic carotid plaques complicated with hemorrhage. Duplex Doppler ultrasonography plaque analysis based on visual in-line classification showed a low level of sensitivity and a moderate-high level of specificity in detecting atherosclerotic carotid plaques complicated with hemorrhage

A method for real-time detection of human fall from video

In this paper we present a method for real-time detection of human fall from video for support of elderly people living alone in their homes. The detection algorithm has four steps: background estimation, extraction of moving objects, motion feature extraction, and fall detection. The detection is based on features that quantify dynamics of human motion and body orientation. The algorithms are implemented in C++ using the OpenCV library. The method is tested using a single camera and 20 test video recordings showing typical fall scenarios and regular household behaviour. The experimental results show 90% of human fall detection accuracy

Predicting creditworthiness in retail banking with limited scoring data

The preoccupation with modelling credit scoring systems including their relevance to predicting and decision making in the financial sector has been with developed countries, whilst developing countries have been largely neglected. The focus of our investigation is on the Cameroonian banking sector with implications for fellow members of the Banque des Etats de L'Afrique Centrale (BEAC) family which apply the same system. We apply logistic regression (LR), Classification and Regression Tree (CART) and Cascade Correlation Neural Network (CCNN) in building our knowledge-based scoring models. To compare various models’ performances, we use ROC curves and Gini coefficients as evaluation criteria and the Kolmogorov-Smirnov curve as a robustness test. The results demonstrate that an improvement in terms of predicting power from 15.69% default cases under the current system, to 7.68% based on the best scoring model, namely CCNN can be achieved. The predictive capabilities of all models are rated as at least very good using the Gini coefficient; and rated excellent using the ROC curve for CCNN. Our robustness test confirmed these results. It should be emphasised that in terms of prediction rate, CCNN is superior to the other techniques investigated in this paper. Also, a sensitivity analysis of the variables identifies previous occupation, borrower's account functioning, guarantees, other loans and monthly expenses as key variables in the forecasting and decision making processes which are at the heart of overall credit policy

« Empowerment », et transfert des acquis dans la remédiation cognitive des personnes atteintes de schizophrénie

Environ 1% de la population française souffre de schizophrénie. Les symptômes liés à leur pathologie les marginalisent. Dans le processus de la réhabilitation sociale, l’ergothérapeute pourra notamment s’appuyer sur la remédiation cognitive. Les ergothérapeutes répondant à une enquête préliminaire ont fait ressortir les notions d’ « empowerment » et de transfert des acquis. Ce mémoire tente à déterminer comment l’ergothérapeute peut associer le transfert des acquis de la remédiation cognitive à l’ « empowerment » des personnes atteintes de schizophrénie.La méthode utilisée pour tenter de répondre à cette question est la méthode clinique par entretien semi-directif. La population ciblée s’est avérée très limitée, rendant les résultats peu exploitables. Néanmoins, la personne interrogée a noté des liens de causalité entre le pouvoir d’agir, l’environnement du patient et le transfert des acquis.Afin d’approfondir la question, il conviendrait d’élargir la population cible et d’étendre la recherche vers une méthode expérimentale

Credit scoring and decision making in Egyptian public sector banks

Purpose – The main aims of this paper are: first, to investigate how decisions are currently made within the Egyptian public sector environment; and, second, to determine whether the decision making can be significantly improved through the use of credit scoring models. A subsidiary aim is to analyze the impact of different proportions of sub-samples of accepted credit applicants on both efficient decision making and the optimal choice of credit scoring techniques. Design/methodology/approach – Following an investigative phase to identify relevant variables in the sector, the research proceeds to an evaluative phase, in which an analysis is undertaken of real data sets (comprising 1,262 applicants), provided by the commercial public sector banks in Egypt. Two types of neural nets are used, and correspondingly two types of conventional techniques are applied. The use of two evaluative measures/criteria: average correct classification (ACC) rate and estimated misclassification cost (EMC) under different misclassification cost (MC) ratios are investigated. Findings – The currently used approach is based on personal judgement. Statistical scoring techniques are shown to provide more efficient classification results than the currently used judgemental techniques. Furthermore, neural net models give better ACC rates, but the optimal choice of techniques depends on the MC ratio. The probabilistic neural net (PNN) is preferred for a lower cost ratio, whilst the multiple discriminant analysis (MDA) is the preferred choice for a higher ratio. Thus, there is a role for MDA as well as neural nets. There is evidence of statistically significant differences between advanced scoring models and conventional models. Research limitations/implications – Future research could investigate the use of further evaluative measures, such as the area under the ROC curve and GINI coefficient techniques and more statistical techniques, such as genetic and fuzzy programming. The plan is to enlarge the data set. Practical implications – There is a huge financial benefit from applying these scoring models to Egyptian public sector banks, for at present only judgemental techniques are being applied in credit evaluation processes. Hence, these techniques can be introduced to support the bank credit decision makers. Originality/value – Thie paper reveals a set of key variables culturally relevant to the Egyptian environment, and provides an evaluation of personal loans in the Egyptian public sector banking environment, in which (to the best of the author's knowledge) no other authors have studied the use of sophisticated statistical credit scoring techniques