Sentic Computing for Aspect-Based Opinion Summarization Using Multi-Head Attention with Feature Pooled Pointer Generator Network

Neural sequence to sequence models have achieved superlative performance in summarizing text. But they tend to generate generic summaries that under-represent the opinion-sensitive aspects of the document. Additionally, the sequence to sequence models are prone to test-train discrepancy (exposure-bias) arising from the differential summary decoding processes in the training and testing phases. The models use ground truth summary words in the decoder training phase and predicted outputs in the testing phase. This inconsistency leads to error accumulation and substandard performance. To address these gaps, a cognitive aspect-based opinion summarizer, Feature Pooled Pointer Generator Network (FP2GN), is proposed which selectively attends to thematic and contextual cues to generate sentiment-aware review summaries. This study augments the pointer generator framework with opinion feature extraction, feature pooling, and mutual attention mechanism for opinion summarization. The proposed model FP2GN identifies the aspect terms in review text using sentic computing (SenticNet 5 and concept frequency-inverse opinion frequency) and statistical feature engineering. These aspect terms are encoded into context embeddings using weighted average feature pooling, which is processed in a pointer-generator framework inspired stacked Bi-LSTM encoder–decoder model with multi-head self-attention. The decoder system uses temporal and mutual attention mechanisms to ensure the appropriate representation of input-sequence. The study also proffers the use of teacher forcing ratio to curtail the exposure-bias-related error-accumulation. The model achieves ROUGE-1 score of 86.04% and ROUGE-L score of 88.51% on the Amazon Fine Foods dataset. An average gain of 2% over other methods is observed. The proposed model reinforces pointer generator network architecture with opinion feature extraction, feature pooling, and mutual attention mechanism to generate human-readable opinion summaries. Empirical analysis substantiates that the proposed model is better than the baseline opinion summarizers

Improving Reader Motivation with Machine Learning

This thesis focuses on the problem of increasing reading motivation with machine learning (ML). The act of reading is central to modern human life, and there is much to be gained by improving the reading experience. For example, the internal reading motivation of students, especially their interest and enjoyment in reading, are important factors in their academic success. There are many topics in natural language processing (NLP) which can be applied to improving the reading experience in terms of readability, comprehension, reading speed, motivation, etc. Such topics include personalized recommendation, headline optimization, text simplification, and many others. However, to the best of our knowledge, this is the first work to explicitly address the broad and meaningful impact that NLP and ML can have on the reading experience. In particular, the aim of this thesis is to explore new approaches to supporting internal reading motivation, which is influenced by readability, situational interest, and personal interest. This is performed by identifying new or existing NLP tasks which can address reader motivation, designing novel machine learning approaches to perform these tasks, and evaluating and examining these approaches to determine what they can teach us about the factors of reader motivation. In executing this research, we make use of concepts from NLP such as textual coherence, interestingness, and summarization. We additionally use techniques from ML including supervised and self-supervised learning, deep neural networks, and sentence embeddings. This thesis, presented in an integrated-article format, contains three core contributions among its three articles. In the first article, we propose a flexible and insightful approach to coherence estimation. This approach uses a new sentence embedding which reflects predicted position distributions. Second, we introduce the new task of pull quote selection, examining a spectrum of approaches in depth. This article identifies several concrete heuristics for finding interesting sentences, both expected and unexpected. Third, we introduce a new interactive summarization task called HARE (Hone as You Read), which is especially suitable for mobile devices. Quantitative and qualitative analysis support the practicality and potential usefulness of this new type of summarization

Optimal Transport in Summarisation: Towards Unsupervised Multimodal Summarisation

Summarisation aims to condense a given piece of information into a short and succinct summary that best covers its semantics with the least redundancy. With the explosion of multimedia data, multimodal summarisation with multimodal output emerges and extends the inquisitiveness of the task. Summarising a video-document pair into a visual-textual summary helps users obtain a more informative and visual understanding. Although various methods have achieved promising performance, they have limitations, including expensive training, lack of interpretability, and insufficient brevity. Therefore, this thesis addresses the gap and examines the application of optimal transport (OT) in unsupervised summarisation. The major contributions are as follows: 1) An interpretable OT-based method is proposed for text summarisation. It formulates summary sentence extraction as minimising the transportation cost of their semantic distributions; 2) An efficient and interpretable unsupervised reinforcement learning method is proposed for text summarisation. Multihead attentional pointer-based networks learn the representation and extract salient sentences and words. The learning strategy mimics human judgment by optimising summary quality regarding OT-based semantic coverage and fluency; 3) A new task, eXtreme Multimodal Summarisation with Multiple Output (XMSMO) is introduced. It summarises a video-document pair into an extremely short multimodal summary. An unsupervised Hierarchical Optimal Transport Network learns and uses OT solvers to maximise multimodal semantic coverage. A new large-scale dataset is constructed to facilitate future research; 4) A Topic-Guided Co-Attention Transformer method is proposed for XMSMO. It constructs a two-stage uni- and cross-modal modelling with topic guidance. An OT-guided unsupervised training strategy optimises the similarity between semantic distributions of topics. Comprehensive experiments demonstrate the effectiveness of the proposed methods

Salience Estimation and Faithful Generation: Modeling Methods for Text Summarization and Generation

This thesis is focused on a particular text-to-text generation problem, automatic summarization, where the goal is to map a large input text to a much shorter summary text. The research presented aims to both understand and tame existing machine learning models, hopefully paving the way for more reliable text-to-text generation algorithms. Somewhat against the prevailing trends, we eschew end-to-end training of an abstractive summarization model, and instead break down the text summarization problem into its constituent tasks. At a high level, we divide these tasks into two categories: content selection, or “what to say” and content realization, or “how to say it” (McKeown, 1985). Within these categories we propose models and learning algorithms for the problems of salience estimation and faithful generation. Salience estimation, that is, determining the importance of a piece of text relative to some context, falls into a problem of the former category, determining what should be selected for a summary. In particular, we experiment with a variety of popular or novel deep learning models for salience estimation in a single document summarization setting, and design several ablation experiments to gain some insight into which input signals are most important for making predictions. Understanding these signals is critical for designing reliable summarization models. We then consider a more difficult problem of estimating salience in a large document stream, and propose two alternative approaches using classical machine learning techniques from both unsupervised clustering and structured prediction. These models incorporate salience estimates into larger text extraction algorithms that also consider redundancy and previous extraction decisions. Overall, we find that when simple, position based heuristics are available, as in single document news or research summarization, deep learning models of salience often exploit them to make predictions, while ignoring the arguably more important content features of the input. In more demanding environments, like stream summarization, where heuristics are unreliable, more semantically relevant features become key to identifying salience content. In part two, content realization, we assume content selection has already been performed and focus on methods for faithful generation (i.e., ensuring that output text utterances respect the semantics of the input content). Since they can generate very fluent and natural text, deep learning- based natural language generation models are a popular approach to this problem. However, they often omit, misconstrue, or otherwise generate text that is not semantically correct given the input content. In this section, we develop a data augmentation and self-training technique to mitigate this problem. Additionally, we propose a training method for making deep learning-based natural language generation models capable of following a content plan, allowing for more control over the output utterances generated by the model. Under a stress test evaluation protocol, we demonstrate some empirical limits on several neural natural language generation models’ ability to encode and properly realize a content plan. Finally, we conclude with some remarks on future directions for abstractive summarization outside of the end-to-end deep learning paradigm. Our aim here is to suggest avenues for constructing abstractive summarization systems with transparent, controllable, and reliable behavior when it comes to text understanding, compression, and generation. Our hope is that this thesis inspires more research in this direction, and, ultimately, real tools that are broadly useful outside of the natural language processing community

Development of an automatic news summarizer for isiXhosa language

From practice perspective, given the abundance of digital content nowadays, coming up with a technological solution that summarizes written text without losing its message, coherence and cohesion of ideas is highly essential. The technology saves time for readers as well as gives them a chance to focus on the contents that matter most. This is one of the research areas in natural language processing/ information retrieval, which the dissertation tries to contribute to. It tries to contextualize tools and technologies that are developed for other languages to automatically summarize textual Xhosa news articles. Specifically, the dissertation aims at developing a text summarizer for textual Xhosa news articles based on the extraction methods. In doing so, it examines the literature and understand the techniques and technologies used to analyse contents of a written text, transform and synthesize it, the phonology and morphology of the Xhosa language, and finally, designs, implements and test an extraction-based automatic news article for the Xhosa language. Given comprehension and relevance of the literature review, the research design, the methods and tools and technologies used to design, implement and test the pilot system. Two approaches were used to extract relevant sentences, which are, term frequency and sentence position. The Xhosa summarizer is evaluated using a test set. This study has employed both subjective and objective evaluation methods. The results of both methods are satisfactory. Keywords: Xhosa, Automatic Text Summarization, Term Frequency and Sentence Position

Management Responses to Online Reviews: Big Data From Social Media Platforms

User-generated content from virtual communities helps businesses develop and sustain competitive advantages, which leads to asking how firms can strategically manage that content. This research, which consists of two studies, discusses management response strategies for hotel firms to gain a competitive advantage and improve customer relationship management by leveraging big data, social media analytics, and deep learning techniques. Since negative reviews' harmful effects are greater than positive comments' contribution, firms must strategise their responses to intervene in and minimise those damages. Although current literature includes a sheer amount of research that presents effective response strategies to negative reviews, they mostly overlook an extensive classification of response strategies. The first study consists of two phases and focuses on comprehensive response strategies to only negative reviews. The first phase is explorative and presents a correlation analysis between response strategies and overall ratings of hotels. It also reveals the differences in those strategies based on hotel class, average customer rating, and region. The second phase investigates effective response strategies for increasing the subsequent ratings of returning customers using logistic regression analysis. It presents that responses involving statements of admittance of mistake(s), specific action, and direct contact requests help increase following ratings of previously dissatisfied returning customers. In addition, personalising the response for better customer relationship management is particularly difficult due to the significant variability of textual reviews with various topics. The second study examines the impact of personalised management responses to positive and negative reviews on rating growth, integrating a novel method of multi-topic matching approach with a panel data analysis. It demonstrates that (a) personalised responses improve future ratings of hotels; (b) the effect of personalised responses is stronger for luxury hotels in increasing future ratings. Lastly, practical insights are provided

Proceedings of the Eighth Italian Conference on Computational Linguistics CliC-it 2021

The eighth edition of the Italian Conference on Computational Linguistics (CLiC-it 2021) was held at Università degli Studi di Milano-Bicocca from 26th to 28th January 2022. After the edition of 2020, which was held in fully virtual mode due to the health emergency related to Covid-19, CLiC-it 2021 represented the first moment for the Italian research community of Computational Linguistics to meet in person after more than one year of full/partial lockdown

Creativity and Wellbeing in Music Education — philosophy, policy and practice in the context of contemporary Scottish primary education

This is an interdisciplinary study encompassing elements of music, education, and aspects of philosophy and psychology. The principal aim of this research is to investigate ways of fostering musical creativity and wellbeing in primary schools, as well as providing curriculum guidelines and drawing out practical implications for Music Education in the present era. Contemporary society represents a new wave of development in human affairs. Resting upon rapid technological innovation, it has created greater opportunities for revealing and experiencing creativity in forms that can then underpin enhanced states of human welfare. At the same time, these far-reaching changes have in many places become threats to human wellbeing, resulting from the dislocating social and emotional impact of new styles of living. As a consequence, ‘creativity’ and ‘wellbeing’ have arisen as important themes in the current era, chiefly as assets and attitudes required for human beings: to live, to respond, to cope, to prosper, and to succeed. In this turbulent context, education is tasked with nurturing both ‘creativity’ and ‘wellbeing’. These concepts are especially meaningful to be investigated through Music Education at the present time, since human beings have had an enduring relationship with sound and music in almost all cultures on record, even and especially those moving through great change. From its basis in musical theory and Music Education, this research also develops a distinctive theoretical foundation in the concepts of ‘Romantic Aesthetics’ and ‘Romantic Irony’––which is a literary, aesthetic, and stylistic term that involves an advanced psychological concept of ‘self’ (e.g. Garber, 2014; Allen, 2007) very apposite to the current age. Specifically, for the present research, I wanted to apply these concepts and theories to the practices of contemporary Music Education, to help devise a useful curriculum for music classes in primary schools consistent with my wider interests in children’s creativity, and children’s wellbeing and resilience, when their lives are often under great pressure. The teaching methods and activities are researched, devised, implemented and evaluated encompassing what is recognised today as the four major components of Music Education: listening to music, singing, playing instruments, and composing. The hypothesis within this research is that applying insights and approaches derived from ‘Romantic Irony’ to Music Education in modern primary schools can also be empowering in fostering pupils’ creativity and wellbeing. Across a broad cross-section of literature in different research areas––not only education but also philosophy and aesthetics, psychology, sociology––it is possible to set the premise that creativity and Romantic Irony are related in various vital aspects. Moreover, it is also possible, this thesis shows, to fashion and actualise a practice of Music Education in regular primary classrooms responsive not only to the rising emphasis on the concept of creativity but also to the pursuit of emotional resilience as a vital and life-supporting dimension of that creativity. Thus, this thesis will attempt to show that applying Romantic Aesthetics and Romantic Irony to Music Education for the development of pupil creativity and wellbeing may a constructive innovation within the compass of all teachers committed to the place of music in the primary curriculum. With due reference to the educational environment and surroundings of Scotland, where this project was deliberately targeted and unfolded, the research herein consists of two types of interventions: a conceptual and an empirical strand. The first part of the research is allocated to investigating and critically assessing theories of creativity, emotion, Romantic Aesthetics, Romantic Irony, Health and Wellbeing, and music therapy––alongside the educational practices in that these concepts may be meaningfully applied or manifest. For the empirical part of the research, I adopted a ‘Vignette’ and ‘thematic approach’ partially indebted to both practioner enquiry and Action Research, to craft ways of enhancing creativity and wellbeing through Music Education in a number of classrooms where I had been previously welcome and active as a serving teacher. The classroom interventions were divided into 3 Vignettes to stimulate pupils’ innate musical creativity and to form relationships, to deliever basic theoretical knowledge, and then to provide opportunities to apply skills in relation to certain topics that appear in daily lives. Thereafter, important academic conversations with experts were conducted in order to examine deeper views of the researcher’s philosophy and approaches and to search for the directions that Music Education ought to follow in contemporary society. The thesis concludes with the conviction that Music Education preserves a rich potential for realising and expressing the core values of progressive education today: promoting for the children in our schools the experiences of creativity, health, resilience and wellbeing which matter so much for surviving and attaining the good life in our protean 21st century society