Georgia concrete pavement performance and longevity

Issued as final reportGeorgia. Dept. of Transportatio

Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains

Pre-trained language models have been applied to various NLP tasks with considerable performance gains. However, the large model sizes, together with the long inference time, limit the deployment of such models in real-time applications. One line of model compression approaches considers knowledge distillation to distill large teacher models into small student models. Most of these studies focus on single-domain only, which ignores the transferable knowledge from other domains. We notice that training a teacher with transferable knowledge digested across domains can achieve better generalization capability to help knowledge distillation. Hence we propose a Meta-Knowledge Distillation (Meta-KD) framework to build a meta-teacher model that captures transferable knowledge across domains and passes such knowledge to students. Specifically, we explicitly force the meta-teacher to capture transferable knowledge at both instance-level and feature-level from multiple domains, and then propose a meta-distillation algorithm to learn single-domain student models with guidance from the meta-teacher. Experiments on public multi-domain NLP tasks show the effectiveness and superiority of the proposed Meta-KD framework. Further, we also demonstrate the capability of Meta-KD in the settings where the training data is scarce

Application of machine learning algorithms to construct and validate a prediction model for coronary heart disease risk in patients with periodontitis: a population-based study

BackgroundThe association between periodontitis and cardiovascular disease is increasingly recognized. In this research, a prediction model utilizing machine learning (ML) was created and verified to evaluate the likelihood of coronary heart disease in individuals affected by periodontitis.MethodsWe conducted a comprehensive analysis of data obtained from the National Health and Nutrition Examination Survey (NHANES) database, encompassing the period between 2009 and 2014.This dataset comprised detailed information on a total of 3,245 individuals who had received a confirmed diagnosis of periodontitis. Subsequently, the dataset was randomly partitioned into a training set and a validation set at a ratio of 6:4. As part of this study, we conducted weighted logistic regression analyses, both univariate and multivariate, to identify risk factors that are independent predictors for coronary heart disease in individuals who have periodontitis. Five different machine learning algorithms, namely Logistic Regression (LR), Gradient Boosting Machine (GBM), Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Classification and Regression Tree (CART), were utilized to develop the model on the training set. The evaluation of the prediction models’ performance was conducted on both the training set and validation set, utilizing metrics including AUC (Area under the receiver operating characteristic curve), Brier score, calibration plot, and decision curve analysis (DCA). Additionally, a graphical representation called a nomogram was created using logistic regression to visually depict the predictive model.ResultsThe factors that were found to independently contribute to the risk, as determined by both univariate and multivariate logistic regression analyses, encompassed age, race, presence of myocardial infarction, chest pain status, utilization of lipid-lowering medications, levels of serum uric acid and serum creatinine. Among the five evaluated machine learning models, the KNN model exhibited exceptional accuracy, achieving an AUC value of 0.977. The calibration plot and brier score illustrated the model's ability to accurately estimate probabilities. Furthermore, the model's clinical applicability was confirmed by DCA.ConclusionOur research showcases the effectiveness of machine learning algorithms in forecasting the likelihood of coronary heart disease in individuals with periodontitis, thereby aiding healthcare professionals in tailoring treatment plans and making well-informed clinical decisions

Qwen Technical Report

Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural language processing tasks that were previously thought to be exclusive to humans. In this work, we introduce Qwen, the first installment of our large language model series. Qwen is a comprehensive language model series that encompasses distinct models with varying parameter counts. It includes Qwen, the base pretrained language models, and Qwen-Chat, the chat models finetuned with human alignment techniques. The base language models consistently demonstrate superior performance across a multitude of downstream tasks, and the chat models, particularly those trained using Reinforcement Learning from Human Feedback (RLHF), are highly competitive. The chat models possess advanced tool-use and planning capabilities for creating agent applications, showcasing impressive performance even when compared to bigger models on complex tasks like utilizing a code interpreter. Furthermore, we have developed coding-specialized models, Code-Qwen and Code-Qwen-Chat, as well as mathematics-focused models, Math-Qwen-Chat, which are built upon base language models. These models demonstrate significantly improved performance in comparison with open-source models, and slightly fall behind the proprietary models.Comment: 59 pages, 5 figure

The Human Activity Radar Challenge: benchmarking based on the ‘Radar signatures of human activities’ dataset from Glasgow University

Radar is an extremely valuable sensing technology for detecting moving targets and measuring their range, velocity, and angular positions. When people are monitored at home, radar is more likely to be accepted by end-users, as they already use WiFi, is perceived as privacy-preserving compared to cameras, and does not require user compliance as wearable sensors do. Furthermore, it is not affected by lighting condi-tions nor requires artificial lights that could cause discomfort in the home environment. So, radar-based human activities classification in the context of assisted living can empower an aging society to live at home independently longer. However, challenges remain as to the formulation of the most effective algorithms for radar-based human activities classification and their validation. To promote the exploration and cross-evaluation of different algorithms, our dataset released in 2019 was used to benchmark various classification approaches. The challenge was open from February 2020 to December 2020. A total of 23 organizations worldwide, forming 12 teams from academia and industry, participated in the inaugural Radar Challenge, and submitted 188 valid entries to the challenge. This paper presents an overview and evaluation of the approaches used for all primary contributions in this inaugural challenge. The proposed algorithms are summarized, and the main parameters affecting their performances are analyzed

The Human Activity Radar Challenge: benchmarking based on the ‘Radar signatures of human activities’ dataset from Glasgow University

Radar is an extremely valuable sensing technology for detecting moving targets and measuring their range, velocity, and angular positions. When people are monitored at home, radar is more likely to be accepted by end-users, as they already use WiFi, is perceived as privacy-preserving compared to cameras, and does not require user compliance as wearable sensors do. Furthermore, it is not affected by lighting condi-tions nor requires artificial lights that could cause discomfort in the home environment. So, radar-based human activities classification in the context of assisted living can empower an aging society to live at home independently longer. However, challenges remain as to the formulation of the most effective algorithms for radar-based human activities classification and their validation. To promote the exploration and cross-evaluation of different algorithms, our dataset released in 2019 was used to benchmark various classification approaches. The challenge was open from February 2020 to December 2020. A total of 23 organizations worldwide, forming 12 teams from academia and industry, participated in the inaugural Radar Challenge, and submitted 188 valid entries to the challenge. This paper presents an overview and evaluation of the approaches used for all primary contributions in this inaugural challenge. The proposed algorithms are summarized, and the main parameters affecting their performances are analyzed

Classification interprétable de séries temporelles

In this thesis, we will study different existing methods that can be used to explain decisions taken by time series classification models. We argue that, in the case of time series, the best explanations should take the form of sub-series (also called shapelets) since it is « pattern language » familiar to a time series user. We review state-of-the-art classification methods that can jointly learn a shapelet-based representation of the series in the dataset and classify the series according to this representation. However, although the learned shapelets are discriminative, they are not always similar to pieces of a real series in the dataset. This makes them difficult to use to explain the classifier’s decision. We make use of a simple convolutional network to tackle the time series classification task and we introduce an adversarial regularization to constrain the model to learn meaningful shapelets. Our classification results, on many univariate time series benchmark datasets, are comparable with the results obtained by state-of-the-art shapelet-based classification algorithms. However, we show, by comparing to other black box explanation methods that our adversarially regularized method learns shapelets that are, by design, better suited to explain decisions.Nous étudions différentes méthodes pouvant être utilisées pour expliquer les décisions prises par les modèles de classification des séries temporelles. Nous supposons que, dans le cas des séries temporelles, les meilleures explications doivent prendre la forme de sous-séries (également appelées shapelets) puisqu'il s'agit d'un "langage" intelligible et expressif pour un utilisateur s'intéressant à ce type de séries. Nous passons en revue les méthodes de classification de l'état de l'art qui peuvent apprendre conjointement une représentation basée sur des shapelets et classer les séries en fonction de cette représentation. Bien que certaines méthodes de l'état de l'art permettent d'apprendre des shapelets discriminantes automatiquement, nous constatons qu'elles ne sont pas toujours similaires aux morceaux d'une série réelle existante. Il est donc difficile de les utiliser pour expliquer la décision du classificateur à un utilisateur qui pourrait être dérouté par ce langage d'explication éloigné des séries qu'il connaît. Nous proposons une méthode innovante qui permet, grâce à un réseau convolutif simple, de classer des séries temporelles et nous introduisons une régularisation antagoniste pour contraindre le modèle à apprendre des shapelets interprétables. Nos résultats de classification sur de nombreux jeux de données de séries temporelles univariées, sont comparables, en terme de précision, aux meilleurs résultats obtenus par les algorithmes de classification basés sur les shapelets. Cependant, nous montrons, en comparant avec d'autres méthodes d'explication sur des modèles de type "boîte noire", que notre régularisation antagoniste permet d'apprendre des shapelets qui sont, par conception, mieux adaptées pour expliquer les décisions et cela pour plusieurs niveaux d'explication

Classification interprétable de séries temporelles

In this thesis, we will study different existing methods that can be used to explain decisions taken by time series classification models. We argue that, in the case of time series, the best explanations should take the form of sub-series (also called shapelets) since it is « pattern language » familiar to a time series user. We review state-of-the-art classification methods that can jointly learn a shapelet-based representation of the series in the dataset and classify the series according to this representation. However, although the learned shapelets are discriminative, they are not always similar to pieces of a real series in the dataset. This makes them difficult to use to explain the classifier’s decision. We make use of a simple convolutional network to tackle the time series classification task and we introduce an adversarial regularization to constrain the model to learn meaningful shapelets. Our classification results, on many univariate time series benchmark datasets, are comparable with the results obtained by state-of-the-art shapelet-based classification algorithms. However, we show, by comparing to other black box explanation methods that our adversarially regularized method learns shapelets that are, by design, better suited to explain decisions.Nous étudions différentes méthodes pouvant être utilisées pour expliquer les décisions prises par les modèles de classification des séries temporelles. Nous supposons que, dans le cas des séries temporelles, les meilleures explications doivent prendre la forme de sous-séries (également appelées shapelets) puisqu'il s'agit d'un "langage" intelligible et expressif pour un utilisateur s'intéressant à ce type de séries. Nous passons en revue les méthodes de classification de l'état de l'art qui peuvent apprendre conjointement une représentation basée sur des shapelets et classer les séries en fonction de cette représentation. Bien que certaines méthodes de l'état de l'art permettent d'apprendre des shapelets discriminantes automatiquement, nous constatons qu'elles ne sont pas toujours similaires aux morceaux d'une série réelle existante. Il est donc difficile de les utiliser pour expliquer la décision du classificateur à un utilisateur qui pourrait être dérouté par ce langage d'explication éloigné des séries qu'il connaît. Nous proposons une méthode innovante qui permet, grâce à un réseau convolutif simple, de classer des séries temporelles et nous introduisons une régularisation antagoniste pour contraindre le modèle à apprendre des shapelets interprétables. Nos résultats de classification sur de nombreux jeux de données de séries temporelles univariées, sont comparables, en terme de précision, aux meilleurs résultats obtenus par les algorithmes de classification basés sur les shapelets. Cependant, nous montrons, en comparant avec d'autres méthodes d'explication sur des modèles de type "boîte noire", que notre régularisation antagoniste permet d'apprendre des shapelets qui sont, par conception, mieux adaptées pour expliquer les décisions et cela pour plusieurs niveaux d'explication