VGCN-BERT : augmenting BERT with graph embedding for text classification : application to offensive language detection

Le discours haineux est un problème sérieux sur les média sociaux. Dans ce mémoire, nous étudions le problème de détection automatique du langage haineux sur réseaux sociaux. Nous traitons ce problème comme un problème de classification de textes. La classification de textes a fait un grand progrès ces dernières années grâce aux techniques d’apprentissage profond. En particulier, les modèles utilisant un mécanisme d’attention tel que BERT se sont révélés capables de capturer les informations contextuelles contenues dans une phrase ou un texte. Cependant, leur capacité à saisir l’information globale sur le vocabulaire d’une langue dans une application spécifique est plus limitée. Récemment, un nouveau type de réseau de neurones, appelé Graph Convolutional Network (GCN), émerge. Il intègre les informations des voisins en manipulant un graphique global pour prendre en compte les informations globales, et il a obtenu de bons résultats dans de nombreuses tâches, y compris la classification de textes. Par conséquent, notre motivation dans ce mémoire est de concevoir une méthode qui peut combiner à la fois les avantages du modèle BERT, qui excelle en capturant des informations locales, et le modèle GCN, qui fournit les informations globale du langage. Néanmoins, le GCN traditionnel est un modèle d'apprentissage transductif, qui effectue une opération convolutionnelle sur un graphe composé d'éléments à traiter dans les tâches (c'est-à-dire un graphe de documents) et ne peut pas être appliqué à un nouveau document qui ne fait pas partie du graphe pendant l'entraînement. Dans ce mémoire, nous proposons d'abord un nouveau modèle GCN de vocabulaire (VGCN), qui transforme la convolution au niveau du document du modèle GCN traditionnel en convolution au niveau du mot en utilisant les co-occurrences de mots. En ce faisant, nous transformons le mode d'apprentissage transductif en mode inductif, qui peut être appliqué à un nouveau document. Ensuite, nous proposons le modèle Interactive-VGCN-BERT qui combine notre modèle VGCN avec BERT. Dans ce modèle, les informations locales captées par BERT sont combinées avec les informations globales captées par VGCN. De plus, les informations locales et les informations globales interagissent à travers différentes couches de BERT, ce qui leur permet d'influencer mutuellement et de construire ensemble une représentation finale pour la classification. Via ces interactions, les informations de langue globales peuvent aider à distinguer des mots ambigus ou à comprendre des expressions peu claires, améliorant ainsi les performances des tâches de classification de textes. Pour évaluer l'efficacité de notre modèle Interactive-VGCN-BERT, nous menons des expériences sur plusieurs ensembles de données de différents types -- non seulement sur le langage haineux, mais aussi sur la détection de grammaticalité et les commentaires sur les films. Les résultats expérimentaux montrent que le modèle Interactive-VGCN-BERT surpasse tous les autres modèles tels que Vanilla-VGCN-BERT, BERT, Bi-LSTM, MLP, GCN et ainsi de suite. En particulier, nous observons que VGCN peut effectivement fournir des informations utiles pour aider à comprendre un texte haiteux implicit quand il est intégré avec BERT, ce qui vérifie notre intuition au début de cette étude.Hate speech is a serious problem on social media. In this thesis, we investigate the problem of automatic detection of hate speech on social media. We cast it as a text classification problem. With the development of deep learning, text classification has made great progress in recent years. In particular, models using attention mechanism such as BERT have shown great capability of capturing the local contextual information within a sentence or document. Although local connections between words in the sentence can be captured, their ability of capturing certain application-dependent global information and long-range semantic dependency is limited. Recently, a new type of neural network, called the Graph Convolutional Network (GCN), has attracted much attention. It provides an effective mechanism to take into account the global information via the convolutional operation on a global graph and has achieved good results in many tasks including text classification. In this thesis, we propose a method that can combine both advantages of BERT model, which is excellent at exploiting the local information from a text, and the GCN model, which provides the application-dependent global language information. However, the traditional GCN is a transductive learning model, which performs a convolutional operation on a graph composed of task entities (i.e. documents graph) and cannot be applied directly to a new document. In this thesis, we first propose a novel Vocabulary GCN model (VGCN), which transforms the document-level convolution of the traditional GCN model to word-level convolution using a word graph created from word co-occurrences. In this way, we change the training method of GCN, from the transductive learning mode to the inductive learning mode, that can be applied to new documents. Secondly, we propose an Interactive-VGCN-BERT model that combines our VGCN model with BERT. In this model, local information including dependencies between words in a sentence, can be captured by BERT, while the global information reflecting the relations between words in a language (e.g. related words) can be captured by VGCN. In addition, local information and global information can interact through different layers of BERT, allowing them to influence mutually and to build together a final representation for classification. In so doing, the global language information can help distinguish ambiguous words or understand unclear expressions, thereby improving the performance of text classification tasks. To evaluate the effectiveness of our Interactive-VGCN-BERT model, we conduct experiments on several datasets of different types -- hate language detection, as well as movie review and grammaticality, and compare them with several state-of-the-art baseline models. Experimental results show that our Interactive-VGCN-BERT outperforms all other models such as Vanilla-VGCN-BERT, BERT, Bi-LSTM, MLP, GCN, and so on. In particular, we have found that VGCN can indeed help understand a text when it is integrated with BERT, confirming our intuition to combine the two mechanisms

Dose-dependent fumonisin B1 hepatotoxicity and hepatocarcinogenicity, detoxification of fumonisin B1, and suppression by isoflavones of fumonisin B1-promoted hepatocarcinogenesis in rats

To determine threshold dose of fumonisin B1 (FB1) hepatotoxicity and hepatocarcinogenicity, rats were initiated with diethylnitrosamine (DEN) and were fed 12.5, 25 and 50 ppm FB1 for 4 weeks. Comparing with control rats, significant increases in plasma alanine aminotransferase activities (ALT) and plasma total cholesterol (TC) were found in animals fed 25 and 50 ppm FB1, and greater hepatic prostaglandin (PG) production in 50 ppm FB1 group. Placental glutathione S-transferase (PGST)- and gamma-glutamyltransferase (GGT)-positive altered hepatic foci (AHF) developed only in 25 and 50 ppm FB1 groups, and feeding 50 ppm FB1 significantly increased the percent areas of PGST- and GGT-positive AHF compared with feeding 25 ppm FB1. The short-term threshold of FB1 hepatotoxicity and hepatocarcinogenicity, therefore, was probably \u3e12.5 ppm but \u3c25 ppm;Fumonisin B1 was reacted with fructose for detoxification. Rats were initiated with DEN and were fed 50 ppm FB1 or 50 ppm FB1 reacted with fructose (FB1-fructose) for 4 weeks. In comparison with the rats fed FB1-fructose, the FB1-fed rats had significantly increased plasma ALT activity, plasma TC, and hepatic PG synthesis. Only in the FB1-fed rats occurred PGST- and GGT-positive AHF. Liver-associated natural killer (NK) cell activity was significantly decreased in the FB1-fed rats and increased in the group fed FB1-fructose. Therefore, reacting FB1 with fructose detoxified FB1 while stimulating the NK cell activity;To evaluate the proposed anticarcinogenic ability of the soy isoflavones genistein and daidzein, DEN-initiated rats were fed 1 mmol total isoflavones/kg diet with 12.5, 25 and 50 ppm FB1 for 4 weeks. Greater plasma TC concentration, ALT activity, hepatic PG production, and development of PGST- and GGT-positive AHF were found in the rats fed diets containing 50 ppm FB1 regardless of isoflavone content. Significantly increased plasma ALT activity, plasma TC, and development of PGST- and GGT- positive AHF were shown in the rats fed 25 ppm FB1. These stimulatory effects of 25 ppm FB1, however, were diminished by isoflavones. Therefore, isoflavones suppressed rat hepatocarcinogenesis promoted by FB1 only at an FB1 dose of 25 ppm and not when 50 ppm was fed

Analysis of Influencing Factors of Tablet Consumer Satisfaction Based on Online Comment Mining

How to extract effective information that affects consumer satisfaction from online comments has become a hot issue for customer behavior. This article is based on the data mining of online comments and the research object are the top-selling tablets on the JD platform from October to December 2018. We started by analyzing influential factors such as goods, after-sales service, and logistics, and crawled online review information of nearly 3,000 tablet computers from five major brands. We first use the jieba word segmentation tool to process the user comments, and use TF-IDF to calculate the frequency of different words in the comments to determine the main keywords of the comments. Secondly, we set up a user\u27s sentiment dictionary to determine the sentiment index of the review, and combined the keywords and sentiment index to get the degree of consumer satisfaction on different influencing factors. Finally, we imported the quantified characteristic factors into Clementine 12.0, and established a Bayesian network model of customer satisfaction, thereby obtaining a ranking table of the importance of each factor to product sales. To improve the model robustness, we adopt a multivariate linear model to check the accuracy of the output results. Our research can not only formulate effective product service sales strategies for merchants, but also guarantee customers to experience better products and services

Investigation of a refrigeration system based on combined supercritical CO2 power and transcritical CO2 refrigeration cycles by waste heat recovery of engine

The majority of the energy in the fuel burned in the internal combustion engines is lost in the form of waste heat. To address this issue, waste heat recovery technology has been proposed to increases the overall efficiency of engine. This paper investigates a heat driven cooling system based on a supercritical CO2 (S-CO2) power cycle integrated with a transcritical CO2 (T-CO2) refrigeration cycle, aiming to provide an alternative to the vapour absorption cooling system. The combined system is proposed to produce cooling for food preservation on a refrigerated truck by waste heat recovery of engine. In this system, the S-CO2 absorbs heat from the exhaust gas and the generated power in the expander is used to drive the compressors in both S-CO2 power cycle and T-CO2 refrigeration cycle. Unlike the bulky vapour absorption cooling system, both power plant and vapour compression refrigerator can be scaled down to a few kilo Watts, opening the possibility for developing small-scale waste heat driven cooling system that can be widely applied for waste heat recovery from IC engines of truck, ship and trains.A new layout sharing a common cooler is also studied. The results suggest that the concept of S-CO2/T-CO2 combined cycle sharing a common cooler has comparable performance and it is thermodynamically feasible. The heat contained in exhaust gas is sufficient for the S-CO2/T-CO2 combined system to provide enough cooling for refrigerated truck cabinet whose surface area is more than 105 m2

Active headrest combined with a depth camera-based ear-positioning system

Active headrests can reduce low-frequency noise around ears based on active noise control (ANC) system. Both the control system using fixed control filters and the remote microphone-based adaptive control system provide good noise reduction performance when the head is in the original position. However, their performance degrades significantly when the head is in motion. In this paper, a human ear-positioning system based on the depth camera is introduced to address this problem. The system uses RTMpose model to estimate the two-dimensional (2D) positions of the ears in the color frame, and then derives the corresponding three-dimensional (3D) coordinates in the depth frame with a depth camera. Experimental results show that the ear-positioning system can effectively track the movement of ears, and the broadband noise reduction performance of the active headrest combined with the system is significantly improved when the human head is translating or rotating

Research on Noise Reduction of Variable Speed Rotary Compressor with Large Capacity

With the increasing speed and capacity of variable-speed rotary compressors, the problem of noise especially low and medium frequency noise in the air conditioning system which can\u27t be solved by wrapping soundproof cotton has became more serious. In this paper, based on the noise problem of the rotor compressor with a working capacity of more than 80CC, the main frequency and the position of the noise source within 1000Hz are confirmed by simulation and experiment. Then on the base of this，the muffler and accumulator are respectively optimized and improved combining with Computer Aided Engineering (CAE) means. The final application results show that the optimized scheme can reduce noise by 6.1dB in 160Hz and 8.9dB in the frequency range of 500Hz to 800Hz, achieving good results

Understanding electronic and optical properties of La and Mn co-doped anatase TiO2

AbstractThe electronic structures, dipole moment, and optical properties of La-doped, Mn-doped, and La-Mn-doped anatase TiO2 have been investigated by means of GGA + U first-principles method, showing that the absorption coefficients of the La-Mn-doped TiO2 under visible light to be sensitive to the doping positions of La and Mn atoms. La-Mn-TiO2(1) exhibits the largest absorption coefficient in visible light because of the smallest band gap. Additionally, for the La and Mn co-doped TiO2, the large dipole moment of TiO6 octahedron is in prejudice of enhancing the optical responses under the visible light. We successfully probe the interesting optical response mechanism in this work