Deep learning with knowledge graphs for fine-grained emotion classification in text

This PhD thesis investigates two key challenges in the area of fine-grained emotion detection in textual data. More specifically, this work focuses on (i) the accurate classification of emotion in tweets and (ii) improving the learning of representations from knowledge graphs using graph convolutional neural networks.The first part of this work outlines the task of emotion keyword detection in tweets and introduces a new resource called the EEK dataset. Tweets have previously been categorised as short sequences or sentence-level sentiment analysis, and it could be argued that this should no longer be the case, especially since Twitter increased its allowed character limit. Recurrent Neural Networks have become a well-established method to classify tweets over recent years, but have struggled with accurately classifying longer sequences due to the vanishing and exploding gradient descent problem. A common technique to overcome this problem has been to prune tweets to a shorter sequence length. However, this also meant that often potentially important emotion carrying information, which is often found towards the end of a tweet, was lost (e.g., emojis and hashtags). As such, tweets mostly face also problems with classifying long sequences, similar to other natural language processing tasks. To overcome these challenges, a multi-scale hierarchical recurrent neural network is proposed and benchmarked against other existing methods. The proposed learning model outperforms existing methods on the same task by up to 10.52%. Another key component for the accurate classification of tweets has been the use of language models, where more recent techniques such as BERT and ELMO have achieved great success in a range of different tasks. However, in Sentiment Analysis, a key challenge has always been to use language models that do not only take advantage of the context a word is used in but also the sentiment it carries. Therefore the second part of this work looks at improving representation learning for emotion classification by introducing both linguistic and emotion knowledge to language models. A new linguistically inspired knowledge graph called RELATE is introduced. Then a new language model is trained on a Graph Convolutional Neural Network and compared against several other existing language models, where it is found that the proposed embedding representations achieve competitive results to other LMs, whilst requiring less pre-training time and data. Finally, it is investigated how the proposed methods can be applied to document-level classification tasks. More specifically, this work focuses on the accurate classification of suicide notes and analyses whether sentiment and linguistic features are important for accurate classification

Modeling of Polish Intonation for Statistical-Parametric Speech Synthesis

Wydział NeofilologiiBieżąca praca prezentuje próbę budowy neurobiologicznie umotywowanego modelu mapowań pomiędzy wysokopoziomowymi dyskretnymi kategoriami lingwistycznymi a ciągłym sygnałem częstotliwości podstawowej w polskiej neutralnej mowie czytanej, w oparciu o konwolucyjne sieci neuronowe. Po krótkim wprowadzeniu w problem badawczy w kontekście intonacji, syntezy mowy oraz luki pomiędzy fonetyką a fonologią, praca przedstawia opis uczenia modelu na podstawie specjalnego korpusu mowy oraz ewaluację naturalności konturu F0 generowanego przez wyuczony model za pomocą eksperymentów percepcyjnych typu ABX oraz MOS przy użyciu specjalnie w tym celu zbudowanego resyntezatora Neural Source Filter. Następnie, prezentowane są wyniki eksploracji fonologiczno-fonetycznych mapowań wyuczonych przez model. W tym celu wykorzystana została jedna z tzw. metod wyjaśniających dla sztucznej inteligencji – Layer-wise Relevance Propagation. W pracy przedstawione zostały wyniki powstałej na tej podstawie obszernej analizy ilościowej istotności dla konturu częstotliwości podstawowej każdej z 1297 specjalnie wygenerowanych lingwistycznych kategorii wejściowych modelu oraz ich wielorakich grupowań na różnorodnych poziomach abstrakcji. Pracę kończy dogłębna analiza oraz interpretacja uzyskanych wyników oraz rozważania na temat mocnych oraz słabych stron zastosowanych metod, a także lista proponowanych usprawnień.This work presents an attempt to build a neurobiologically inspired Convolutional Neural Network-based model of the mappings between discrete high-level linguistic categories into a continuous signal of fundamental frequency in Polish neutral read speech. After a brief introduction of the current research problem in the context of intonation, speech synthesis and the phonetic-phonology gap, the work goes on to describe the training of the model on a special speech corpus, and an evaluation of the naturalness of the F0 contour produced by the trained model through ABX and MOS perception experiments conducted with help of a specially built Neural Source Filter resynthesizer. Finally, an in-depth exploration of the phonology-to-phonetics mappings learned by the model is presented; the Layer-wise Relevance Propagation explainability method was used to perform an extensive quantitative analysis of the relevance of 1297 specially engineered linguistic input features and their groupings at various levels of abstraction for the specific contours of the fundamental frequency. The work ends with an in-depth interpretation of these results and a discussion of the advantages and disadvantages of the current method, and lists a number of potential future improvements.Badania przedstawione w pracy zostały cz˛e´sciowo zrealizowane w ramach grantu badawczego Harmonia nr UMO-2014/14/M/HS2/00631 przyznanego przez Narodowe Centrum Nauki

Natural Language Processing: Emerging Neural Approaches and Applications

This Special Issue highlights the most recent research being carried out in the NLP field to discuss relative open issues, with a particular focus on both emerging approaches for language learning, understanding, production, and grounding interactively or autonomously from data in cognitive and neural systems, as well as on their potential or real applications in different domains

Towards hypertextual music

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonThis is a study of the way in which digital audio and a number of key associated technologies that rely on it as a framework have changed the creation, production and dissemination of music, as witnessed by my own creative practice. The study is built on my own work as an electronic musician and composer and draws from numerous collaborations with not only other musicians but also researchers and artists, as documented through commissions, performances, academic papers and commercial releases over an 9 year period from 2007 to 2016. I begin by contextualising my own musical practice and outlining some prominent themes associated with the democratisation of computing that the work of this thesis interrogates as a critical framework for the production of musical works. I go on to assess how works using various techniques afforded by digital audio may be interpreted as progressively instantiating a digital ontology of music. In the context of this digital ontology of music I propose a method of analysis and criticism of works explicitly concerned with audio analysis and algorithmic processes based on my interpretation of the concept of `hypertext', wherein the ability for computers to analyse, index and create multi-dimensional, non-linear links between segments of digital audio is best described as hypertextual. In light of this, I contextualise the merits of this reading of music created using these affordances of digital audio through a reading of several key works of 20th century music from a hypertextual perspective, emphasising the role information theory and semiotics have to play in analyses of these works. I proffer this as the beginnings of a useful model for musical composition in the domain of digital audio which I seek to explore through my own practice. I then describe and analyse, both individually and in parallel numerous works I have undertaken that seek to interrogate the intricacies of what it means to work in the domain of digital audio with audio analysis, machine listening, algorithmic and generative computational processes and consider the ways in which aspects of this work might be seen as contributing useful and novel insights into music creation by harnessing properties intrinsic to digital audio as a medium. Finally, I emphasise, based on the music and research presented in the thesis, the extent to which digital audio and the harnessing of increasingly complex computational systems for the production and dissemination of music has changed the ontology of music production, a situation which I interpret as creating both substantial challenges, but also great possibilities for the future of music

What can a sonic assemblage do? A biopsychosocial approach to post-acousmatic composition

Thinking and sounding are two terms which complicate one another, hence this thesis follows two trajectories each of which make an original contribution to knowledge. Part 1 (thinking sound) proposes to reground composition away from historically authoritative humanist models, instead suggesting a biopsychosocial approach for a post-acousmatic music. I elaborate a set of models and key concepts, chiefly an eliminativist account of the listener-sound relation; neurocognitively discrete musical domains and dimensions of the Kmatrix; model-based reasoning through a Reception-Interpretation-Action helix; and, mentalizing listening stances based upon dual-process cognition models. This is combined with an art-activist stance where composition is concerned with the effects that a sonic artobject exerts in its vicinity. I propose composition as experimentally concerned with generating new epistemic things through a process of assemblage and heterogeneous engineering. Part 2 (sounding thinking) discusses fixed and live compositions which initiated and respond to my proposed approach. In my practice, I focus on the disruption of specific aesthetic regimens to bring listening into attentional focus, engaging the specificity of the mnemonic traces that sound leaves. The pieces are largely concerned with sonic cultures related to Islam and the MENASA region

Intelligent Sensors for Human Motion Analysis

The book, "Intelligent Sensors for Human Motion Analysis," contains 17 articles published in the Special Issue of the Sensors journal. These articles deal with many aspects related to the analysis of human movement. New techniques and methods for pose estimation, gait recognition, and fall detection have been proposed and verified. Some of them will trigger further research, and some may become the backbone of commercial systems