Toxic comment classification using convolutional and recurrent neural networks

This thesis aims to provide a reasonable solution for categorizing automatically sentences into types of toxicity using different types of neural networks. There are six types of categories: Toxic, severe toxic, obscene, threat, insult and identity hate. Three different implementations have been studied to accomplish the objective: LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit) and convolutional neural networks. The thesis is not thought to aim on improving the performance of every individual model but on the comparison between them in terms of natural language processing adequacy. In addition, one differential aspect about this project is the research of LSTM neurons activations and thus the relationship of the words with the final sentence classificatory decision. In conclusion, the three models performed almost equally and the extraction of LSTM activations provided a very accurate and visual understanding of the decisions taken by the network.Esta tesis tiene como objetivo aportar una buena solución para la categorización automática de comentarios abusivos haciendo uso de distintos tipos de redes neuronales. Hay seis categorías: Tóxico, muy tóxico, obsceno, insulto, amenaza y racismo. Se ha hecho una investigación de tres implementaciones para llevar a cabo el objetivo: LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit) y redes convolucionales. El objetivo de este trabajo no es intentar mejorar al máximo el resultado de la clasificación sino hacer una comparación de los 3 modelos para los mismos parámetros e intentar saber cuál funciona mejor para este caso de procesado de lenguaje. Además, un aspecto diferencial de este proyecto es la investigación sobre las activaciones de las neuronas en el modelo LSTM y su relación con la importancia de las palabras respecto a la clasificación final de la frase. En conclusión, los tres modelos han funcionado de forma casi idéntica y la extracción de las activaciones han proporcionado un conocimiento muy preciso y visual de las decisiones tomadas por la red.Aquesta tesi té com a objectiu aportar una bona solució per categoritzar automàticament comentaris abusius usant diferents tipus de xarxes neuronals. Hi ha sis tipus de categories: Tòxic, molt tòxic, obscè, insult, amenaça i racisme. S'ha fet una recerca de tres implementacions per dur a terme l'objectiu: LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit) i xarxes convolucionals. L'objectiu d'aquest treball no és intentar millorar al màxim els resultats de classificació sinó fer una comparació dels 3 models pels mateixos paràmetres per tal d'esbrinar quin funciona millor en aquest cas de processat de llenguatge. A més, un aspecte diferencial d'aquest projecte és la recerca sobre les activacions de les neurones al model LSTM i la seva relació amb la importància de les paraules respecte la classificació final de la frase. En conclusió, els tres models han funcionat gairebé idènticament i l'extracció de les activacions van proporcionar un enteniment molt acurat i visual de les decisions preses per la xarxa

Survey on software tools that implement deep learning algorithms on intel/x86 and IBM/Power8/Power9 platforms

Neural networks are becoming more and more popular in scientific field and in the industry. It is mostly because new solutions using neural networks show state-of-the-art results in the domains previously occupied by traditional methods, eg. computer vision, speech recognition etc. But to get these results neural networks become progressively more complex, thus needing a lot more training. The training of neural networks today can take weeks. This problems can be solved by parallelization of the neural networks training and using modern clusters and supercomputers, which can significantly reduce the learning time. Today, a faster training for data scientist is essential, because it allows to get the results faster to make the next decision. In this paper we provide an overview of distributed learning provided by the popular modern deep learning frameworks, both in terms of provided functionality and performance. We consider multiple hardware choices: training on multiple GPUs and multiple computing nodes. © The Authors 2019.Council on grants of the President of the Russian Federation: MK-2330.2019.9You can use a special version of Caffe, NVCaffe, which is supported by NVidia. This version was created specifically for the use of several GPUs. User instructions can be found in [35].For NVidia, MXNet is supported by Nvidia Cloud. MXNet also has support for CUDA and CuDNN.The results described in this paper were obtained with the financial support of the grant from the Russian Federation President Fund (MK-2330.2019.9)