Comprehensive Review of Opinion Summarization

The abundance of opinions on the web has kindled the study of opinion summarization over the last few years. People have introduced various techniques and paradigms to solving this special task. This survey attempts to systematically investigate the different techniques and approaches used in opinion summarization. We provide a multi-perspective classification of the approaches used and highlight some of the key weaknesses of these approaches. This survey also covers evaluation techniques and data sets used in studying the opinion summarization problem. Finally, we provide insights into some of the challenges that are left to be addressed as this will help set the trend for future research in this area.unpublishednot peer reviewe

Text-image synergy for multimodal retrieval and annotation

Text and images are the two most common data modalities found on the Internet. Understanding the synergy between text and images, that is, seamlessly analyzing information from these modalities may be trivial for humans, but is challenging for software systems. In this dissertation we study problems where deciphering text-image synergy is crucial for finding solutions. We propose methods and ideas that establish semantic connections between text and images in multimodal contents, and empirically show their effectiveness in four interconnected problems: Image Retrieval, Image Tag Refinement, Image-Text Alignment, and Image Captioning. Our promising results and observations open up interesting scopes for future research involving text-image data understanding.Text and images are the two most common data modalities found on the Internet. Understanding the synergy between text and images, that is, seamlessly analyzing information from these modalities may be trivial for humans, but is challenging for software systems. In this dissertation we study problems where deciphering text-image synergy is crucial for finding solutions. We propose methods and ideas that establish semantic connections between text and images in multimodal contents, and empirically show their effectiveness in four interconnected problems: Image Retrieval, Image Tag Refinement, Image-Text Alignment, and Image Captioning. Our promising results and observations open up interesting scopes for future research involving text-image data understanding.Text und Bild sind die beiden häufigsten Arten von Inhalten im Internet. Während es für Menschen einfach ist, gerade aus dem Zusammenspiel von Text- und Bildinhalten Informationen zu erfassen, stellt diese kombinierte Darstellung von Inhalten Softwaresysteme vor große Herausforderungen. In dieser Dissertation werden Probleme studiert, für deren Lösung das Verständnis des Zusammenspiels von Text- und Bildinhalten wesentlich ist. Es werden Methoden und Vorschläge präsentiert und empirisch bewertet, die semantische Verbindungen zwischen Text und Bild in multimodalen Daten herstellen. Wir stellen in dieser Dissertation vier miteinander verbundene Text- und Bildprobleme vor: • Bildersuche. Ob Bilder anhand von textbasierten Suchanfragen gefunden werden, hängt stark davon ab, ob der Text in der Nähe des Bildes mit dem der Anfrage übereinstimmt. Bilder ohne textuellen Kontext, oder sogar mit thematisch passendem Kontext, aber ohne direkte Übereinstimmungen der vorhandenen Schlagworte zur Suchanfrage, können häufig nicht gefunden werden. Zur Abhilfe schlagen wir vor, drei Arten von Informationen in Kombination zu nutzen: visuelle Informationen (in Form von automatisch generierten Bildbeschreibungen), textuelle Informationen (Stichworte aus vorangegangenen Suchanfragen), und Alltagswissen. • Verbesserte Bildbeschreibungen. Bei der Objekterkennung durch Computer Vision kommt es des Öfteren zu Fehldetektionen und Inkohärenzen. Die korrekte Identifikation von Bildinhalten ist jedoch eine wichtige Voraussetzung für die Suche nach Bildern mittels textueller Suchanfragen. Um die Fehleranfälligkeit bei der Objekterkennung zu minimieren, schlagen wir vor Alltagswissen einzubeziehen. Durch zusätzliche Bild-Annotationen, welche sich durch den gesunden Menschenverstand als thematisch passend erweisen, können viele fehlerhafte und zusammenhanglose Erkennungen vermieden werden. • Bild-Text Platzierung. Auf Internetseiten mit Text- und Bildinhalten (wie Nachrichtenseiten, Blogbeiträge, Artikel in sozialen Medien) werden Bilder in der Regel an semantisch sinnvollen Positionen im Textfluss platziert. Wir nutzen dies um ein Framework vorzuschlagen, in dem relevante Bilder ausgesucht werden und mit den passenden Abschnitten eines Textes assoziiert werden. • Bildunterschriften. Bilder, die als Teil von multimodalen Inhalten zur Verbesserung der Lesbarkeit von Texten dienen, haben typischerweise Bildunterschriften, die zum Kontext des umgebenden Texts passen. Wir schlagen vor, den Kontext beim automatischen Generieren von Bildunterschriften ebenfalls einzubeziehen. Üblicherweise werden hierfür die Bilder allein analysiert. Wir stellen die kontextbezogene Bildunterschriftengenerierung vor. Unsere vielversprechenden Beobachtungen und Ergebnisse eröffnen interessante Möglichkeiten für weitergehende Forschung zur computergestützten Erfassung des Zusammenspiels von Text- und Bildinhalten

A survey of data mining techniques for social media analysis

Social network has gained remarkable attention in the last decade. Accessing social network sites such as Twitter, Facebook LinkedIn and Google+ through the internet and the web 2.0 technologies has become more affordable. People are becoming more interested in and relying on social network for information, news and opinion of other users on diverse subject matters. The heavy reliance on social network sites causes them to generate massive data characterised by three computational issues namely; size, noise and dynamism. These issues often make social network data very complex to analyse manually, resulting in the pertinent use of computational means of analysing them. Data mining provides a wide range of techniques for detecting useful knowledge from massive datasets like trends, patterns and rules [44]. Data mining techniques are used for information retrieval, statistical modelling and machine learning. These techniques employ data pre-processing, data analysis, and data interpretation processes in the course of data analysis. This survey discusses different data mining techniques used in mining diverse aspects of the social network over decades going from the historical techniques to the up-to-date models, including our novel technique named TRCM. All the techniques covered in this survey are listed in the Table.1 including the tools employed as well as names of their authors

Integrating Deep Contextualized Word Embeddings into Text Summarization Systems

In questa tesi saranno usate tecniche di deep learning per affrontare unodei problemi più difficili dell’elaborazione automatica del linguaggio naturale:la generazione automatica di riassunti. Dato un corpus di testo, l’obiettivoè quello di generare un riassunto che sia in grado di distillare e comprimerel’informazione dall’intero testo di partenza. Con i primi approcci si é provatoa catturare il significato del testo attraverso l’uso di regole scritte dagliumani. Dopo questa era simbolica basata su regole, gli approcchi statistici hanno preso il sopravvento. Negli ultimi anni il deep learning ha impattato positivamente ogni area dell’elaborazione automatica del linguaggionaturale, incluso la generazione automatica dei riassunti. In questo lavoroi modelli pointer-generator [See et al., 2017] sono utilizzati in combinazionea pre-trained deep contextualized word embeddings [Peters et al., 2018]. Sivaluta l’approccio sui due più grossi dataset per la generazione automaticadei riassunti disponibili ora: il dataset CNN/Daily Mail e il dataset Newsroom. Il dataset CNN/Daily Mail è stato generato partendo dal dataset diQuestion Answering pubblicato da DeepMind [Hermann et al., 2015], concatenando le frasi di highlight delle news e formando cosı̀ dei riassunti multifrase. Il dataset Newsroom [Grusky et al., 2018] è, invece, il primo datasetesplicitamente costruito per la generazione automatica di riassunti. Comprende un milione di coppie articolo-riassunto con diversi gradi di estrattività/astrattività a diversi ratio di compressione.L’approccio è valutato sui test-set con l’uso della metrica Recall-Oriented Understudy for Gisting Evaluation (ROUGE). Questo approccio causa un sostanzioso aumento nelle performance per il dataset Newsroom raggiungendo lo stato dell’arte sul valore di ROUGE-1 e valori competitivi per ROUGE-2 e ROUGE-L

ChatGPT Application In Summarizing An Evolution Of Deep Learning Techniques In Imaging: A Qualitative Study

The pursuit of article or text summarization has captured the attention of natural language processing (NLP) practitioners, presenting itself as a formidable challenge. ChatGPT 3.5 exhibits the capacity to condense the content of up to 3000 tokens into a single page, aiming to retain pivotal information from a given text across diverse themes. In a conducted qualitative research endeavor, we selected seven scientific articles and employed the publicly available ChatGPT service to generate summaries of these articles. Subsequently, we engaged six co-authors of the articles in a survey, presenting five questions to evaluate the quality of the summaries compared to the original content. The findings revealed that the summaries produced by ChatGPT effectively encapsulated the crucial information present in the articles, preserving the principal message of each manuscript. Nonetheless, there was a slight diminishment in the technical depth of the summaries as opposed to the original articles. As a result, our conclusion underscores ChatGPT's text summarization capability as a potent tool for extracting essential insights in a manner more aligned with reporting than purely scientific discourse

Adapting Automatic Summarization to New Sources of Information

English-language news articles are no longer necessarily the best source of information. The Web allows information to spread more quickly and travel farther: first-person accounts of breaking news events pop up on social media, and foreign-language news articles are accessible to, if not immediately understandable by, English-speaking users. This thesis focuses on developing automatic summarization techniques for these new sources of information. We focus on summarizing two specific new sources of information: personal narratives, first-person accounts of exciting or unusual events that are readily found in blog entries and other social media posts, and non-English documents, which must first be translated into English, often introducing translation errors that complicate the summarization process. Personal narratives are a very new area of interest in natural language processing research, and they present two key challenges for summarization. First, unlike many news articles, whose lead sentences serve as summaries of the most important ideas in the articles, personal narratives provide no such shortcuts for determining where important information occurs in within them; second, personal narratives are written informally and colloquially, and unlike news articles, they are rarely edited, so they require heavier editing and rewriting during the summarization process. Non-English documents, whether news or narrative, present yet another source of difficulty on top of any challenges inherent to their genre: they must be translated into English, potentially introducing translation errors and disfluencies that must be identified and corrected during summarization. The bulk of this thesis is dedicated to addressing the challenges of summarizing personal narratives found on the Web. We develop a two-stage summarization system for personal narrative that first extracts sentences containing important content and then rewrites those sentences into summary-appropriate forms. Our content extraction system is inspired by contextualist narrative theory, using changes in writing style throughout a narrative to detect sentences containing important information; it outperforms both graph-based and neural network approaches to sentence extraction for this genre. Our paraphrasing system rewrites the extracted sentences into shorter, standalone summary sentences, learning to mimic the paraphrasing choices of human summarizers more closely than can traditional lexicon- or translation-based paraphrasing approaches. We conclude with a chapter dedicated to summarizing non-English documents written in low-resource languages – documents that would otherwise be unreadable for English-speaking users. We develop a cross-lingual summarization system that performs even heavier editing and rewriting than does our personal narrative paraphrasing system; we create and train on large amounts of synthetic errorful translations of foreign-language documents. Our approach produces fluent English summaries from disdisfluent translations of non-English documents, and it generalizes across languages

The intelligence in the loop

For decades, engineering in computing systems has used a human-in-the-loop servo mechanism. A conscious human being is usually believed, in a rational manner, to operate, assist, and control the machine to achieve desired objectives. Over time, researchers have started to use human-in-the-loop schemes in more abstract tasks, such as iterative interface design problems. However, with the observations and developments in social science, the underlying rationality assumption is strongly challenged, and humans make mistakes. With the recent advances in computer science regarding artificial intelligence, data-driven algorithms could achieve human-level performance in certain aspects, such as audio recognition, image segmentation, and machine translation tasks. The human-in-the-loop mechanism is being reconsidered and reshaped towards an extended vision to assist human decision-making or creativity in the human-computer interaction (HCI) research field. This thesis explores the boundary for human-in-the-loop optimization systems to succeed and be beneficial. In the interaction loop, machine agents are designed rationally to interact with human beings that may behave using incomplete rational policies iteratively. The thesis first examines and deliberates common principles in mainstream HCI research regarding the advice for building human-in-the-loop systems using existing computation techniques concerning decision-making support, utility-based optimization, and human concepts regarding preferences, satisfaction, and expertise. To reflect real-world constraints in a human-in-the-loop optimization system, the thesis explores three design problems: text summarization, image color enhancement, and 3D polygon reduction. These design problems are selected to involve human perception and intelligence, aesthetic preference, and rational judgments. Specifically, to understand and analyze the interaction loop, the thesis conducted a series of experiments to study the impact of various building blocks in human-in-the-loop systems that observes exploration and exploitation of human users, including problem context, solution space, reliability of human inputs regarding preference and expertise, and relevant user interfaces for inputs. Combining the findings of the experiments, the thesis revisits vulnerable assumptions that may be largely ignored when designing a modern human-in-the-loop optimization system. The experiment on the impact of user interfaces narrows down the exploration space of this thesis and empirically demonstrates how different preferential user interfaces influence the overall interaction performance. Based on the findings, subsequent experiments further investigate how human judgments can be a flaw of a human-in-the-loop optimization system. The result shows that, due to cognitive limitations and unrealistic system assumptions, inconsistent and unstable preferences commonly exist in this human-in-the-loop optimization system, resulting in suboptimal machine outcomes and user dissatisfaction, which conflicts with the objective of using a human to gain the expected output. With a deeper look into human aspects, another experiment attempts to reveal the potential causes, such as involved level of human expertise. The system further tests the usage of individuals with different levels of expertise. Based on the observation and analysis, higher-level expertise leads to lower subjective satisfaction and more interactions, whereas novices terminate faster and also achieve expert-level performance, which not only reveals challenges to utilizing the obtained human insights but also be considered as an indicator to reveal how we can better involve a human in an optimization loop for exploring a solution space. All these contributions in human-in-the-loop optimization systems lead to a rethinking of the source of intelligence and engage philosophical discussions. These topics eventually approach more fundamental questions regarding the definition of intelligence and how we might succeed in keeping our intelligence in the loop.Seit Jahrzehnten verwendet das Ingenieurwesen in Computersystemen einen "human-in-the-loop" Servomechanismus. Ein bewusster Mensch wird in der Regel auf rationale Weise eingesetzt, um die Maschine zu bedienen, zu unterstützen und zu kontrollieren, um die gewünschten Ziele zu erreichen. Im Laufe der Zeit haben Forscher begonnen, "human-in-the-loop" Schemata in abstrakteren Aufgabenstellungen wie iterativen Schnittstellendesignproblemen einzusetzen. Allerdings wird mit den Beobachtungen und Entwicklungen in den Sozialwissenschaften die zugrunde liegende Rationalitätsannahme stark in Frage gestellt und Menschen machen Fehler. Mit den jüngsten Fortschritten in der Informatik im Bereich der künstlichen Intelligenz könnten datengetriebene Algorithmen in bestimmten Bereichen menschenähnliche Leistungen erbringen, wie zum Beispiel bei der Audioerkennung, Bildsegmentierung und maschinellen Übersetzung. Der "human-in-the-loop" Mechanismus wird im Bereich der Forschung zur Mensch-Computer-Interaktion (MCI) neu überdacht und neu gestaltet, um die menschliche Entscheidungsfindung oder Kreativität zu unterstützen. Diese Arbeit untersucht die Grenzen für "human-in-the-loop" Optimierungssysteme, um erfolgreich und vorteilhaft zu sein. In der Interaktionsschleife werden Maschinenagenten rational entworfen, um mit menschlichen Wesen zu interagieren, die iterativ möglicherweise mit unvollständigen rationalen Richtlinien handeln. Die Arbeit untersucht und diskutiert zunächst gemeinsame Prinzipien in der Mainstream-Forschung zur Mensch-Computer-Interaktion (MCI) hinsichtlich der Empfehlungen für den Aufbau von "human-in-the-loop" Systemen unter Verwendung vorhandener Berechnungstechniken zur Entscheidungsunterstützung, nutzungsbasierter Optimierung und menschlichen Konzepten bezüglich Vorlieben, Zufriedenheit und Expertise. Um realitätsnahe Einschränkungen in einem "human-in-the-loop" Optimierungssystem widerzuspiegeln, untersucht die Arbeit drei Designprobleme: Textzusammenfassung, Verbesserung von Bildfarben und Reduzierung von 3D-Polygonen. Diese Designprobleme wurden ausgewählt, um die menschliche Wahrnehmung und Intelligenz, ästhetische Präferenzen und rationale Urteile einzubeziehen. Um die Interaktionsschleife zu verstehen und zu analysieren, führte die Arbeit eine Reihe von Experimenten durch, um die Auswirkungen verschiedener Bausteine in "human-in-the-loop" Systemen zu untersuchen, die die Exploration und Ausnutzung menschlicher Benutzer berücksichtigen, einschließlich des Problemkontexts, des Lösungsraums, der Zuverlässigkeit menschlicher Eingaben bezüglich Vorlieben und Expertise sowie relevanter Benutzeroberflächen für Eingaben. Durch die Kombination der Ergebnisse der Experimente hinterfragt die Arbeit anfällige Annahmen, die bei der Gestaltung eines modernen "human-in-the-loop" Optimierungssystems weitgehend ignoriert werden können. Das Experiment zur Auswirkung von Benutzeroberflächen begrenzt den Explorationsspielraum dieser Arbeit und zeigt empirisch, wie unterschiedliche bevorzugte Benutzeroberflächen die Gesamtleistung der Interaktion beeinflussen. Basierend auf den Ergebnissen untersuchen nachfolgende Experimente weiter, wie menschliche Urteile die Schwachstelle eines “human-in-the-loop” Optimierungssystems werden können. Das Ergebnis zeigt, dass aufgrund kognitiver Einschränkungen und unrealistischer Systemannahmen inkonsistente und instabile Präferenzen in diesem "human-in-the-loop" Optimierungssystem häufig vorkommen und zu suboptimalen Maschinenergebnissen und Benutzerunzufriedenheit führen, was dem Ziel widerspricht, einen Menschen zur Erzielung des erwarteten Outputs zu nutzen. Mit einem tieferen Blick auf menschliche Aspekte versucht ein weiteres Experiment, potenzielle Ursachen aufzudecken, wie zum Beispiel das involvierte Niveau menschlicher Expertise. Das System testet außerdem die Verwendung von Personen mit unterschiedlichen Kenntnisständen. Basierend auf Beobachtungen und Analysen führt höhere Expertise zu geringerer subjektiver Zufriedenheit und mehr Interaktionen, während Anfänger schneller aufgeben und auch eine Expertenleistung erbringen. Dies zeigt nicht nur Herausforderungen bei der Nutzung der gewonnenen menschlichen Erkenntnisse auf, sondern kann auch als Indikator dienen, um aufzuzeigen, wie wir einen Menschen besser in eine Optimierungsschleife einbeziehen können, um einen Lösungsraum zu erkunden. All diese Beiträge in "human-in-the-loop" Optimierungssystemen führen zu einem Umdenken über die Quelle der Intelligenz und führen zu philosophischen Diskussionen. Diese Themen nähern sich schließlich grundlegenderen Fragen über die Definition von Intelligenz und wie es uns gelingen könnte, unsere intelligence in the loop zu halten

Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges

Generative Artificial Intelligence (AI) is one of the most exciting developments in Computer Science of the last decade. At the same time, Reinforcement Learning (RL) has emerged as a very successful paradigm for a variety of machine learning tasks. In this survey, we discuss the state of the art, opportunities and open research questions in applying RL to generative AI. In particular, we will discuss three types of applications, namely, RL as an alternative way for generation without specified objectives; as a way for generating outputs while concurrently maximizing an objective function; and, finally, as a way of embedding desired characteristics, which cannot be easily captured by means of an objective function, into the generative process. We conclude the survey with an in-depth discussion of the opportunities and challenges in this fascinating emerging area.Comment: Published in JAIR at https://www.jair.org/index.php/jair/article/view/1527