Leveraging Key Information Modeling to Improve Less-Data Constrained News Headline Generation via Duality Fine-Tuning

Recent language generative models are mostly trained on large-scale datasets, while in some real scenarios, the training datasets are often expensive to obtain and would be small-scale. In this paper we investigate the challenging task of less-data constrained generation, especially when the generated news headlines are short yet expected by readers to keep readable and informative simultaneously. We highlight the key information modeling task and propose a novel duality fine-tuning method by formally defining the probabilistic duality constraints between key information prediction and headline generation tasks. The proposed method can capture more information from limited data, build connections between separate tasks, and is suitable for less-data constrained generation tasks. Furthermore, the method can leverage various pre-trained generative regimes, e.g., autoregressive and encoder-decoder models. We conduct extensive experiments to demonstrate that our method is effective and efficient to achieve improved performance in terms of language modeling metric and informativeness correctness metric on two public datasets.Comment: Accepted by AACL-IJCNLP 2022 main conferenc

Street cred : are media consumers craving more “authenticity” in the digital age?

Media organizations that provide news have traditionally relied on audience perceptions of truth and credibility to lure more readers, viewers, and listeners. The author explores whether authenticity has emerged in the digital age as an additional ingredient in media consumers’ daily decisions about where to turn – and where to return – for trustworthy information. As it becomes ever-more of a challenge for consumers to distinguish reliable information from “fake news” in the 21st century, audiences may be seeking content from media organizations that feels more authentic, genuine, and personalized. Three case studies drawn from new media, as well as legacy media, help illustrate what traditional and startup media institutions can do to better understand audience attitudes and behaviors: the HBO series “Last Week Tonight with John Oliver,” the online site Reddit, and the Pulitzer Prize-winning work of Washington Post reporter David Fahrenthold.Communication Studie

Automatic Generation of Factual News Headlines in Finnish

Peer reviewe

Overcoming the Age Barrier: Improving Older Adults’ Detection of Political Disinformation With Media Literacy

This experimental study analyzes the effect of media literacy on the ability of Spanish seniors over 50 years of age to identify fake news. The experiment measures the improvement achieved by older adults in the detection of political disinformation thanks to a digital competence course offered through WhatsApp. The study comprises a total sample of 1,029 individuals, subdivided into a control group (n = 531) and an experimental group (n = 498), from which a qualified experimental subsample (n = 87) was extracted. Results reveal that participants’ political beliefs, ranging from left to right positions, influence their ability to detect misinformation. A progressive political position is associated with higher accuracy in identifying right-biased news headlines and lower accuracy for left-biased headlines. A conservative position is associated with higher accuracy when the news headline has a progressive bias, but lower accuracy when the headline is right-wing. Users are more critical when the headline has a bias against theirs, while they are more likely to believe news that confirms their own beliefs. The study adds evidence on the relevance of cognitive biases in disinformation and supports the convenience of designing specific media literacy actions aimed at older adults

Insights into Classifying and Mitigating LLMs' Hallucinations

The widespread adoption of large language models (LLMs) across diverse AI applications is proof of the outstanding achievements obtained in several tasks, such as text mining, text generation, and question answering. However, LLMs are not exempt from drawbacks. One of the most concerning aspects regards the emerging problematic phenomena known as "Hallucinations". They manifest in text generation systems, particularly in question-answering systems reliant on LLMs, potentially resulting in false or misleading information propagation. This paper delves into the underlying causes of AI hallucination and elucidates its significance in artificial intelligence. In particular, Hallucination classification is tackled over several tasks (Machine Translation, Question and Answer, Dialog Systems, Summarisation Systems, Knowledge Graph with LLMs, and Visual Question Answer). Additionally, we explore potential strategies to mitigate hallucinations, aiming to enhance the overall reliability of LLMs. Our research addresses this critical issue within the HeReFaNMi (Health-Related Fake News Mitigation) project, generously supported by NGI Search, dedicated to combating Health-Related Fake News dissemination on the Internet. This endeavour represents a concerted effort to safeguard the integrity of information dissemination in an age of evolving AI technologies.Comment: Accepted at AIxIA 202

Approach to transforming training data for improving the title generation performance for scientific texts

Предлагается подход к улучшению качества генерации заголовков, основанный на ранжировании примеров обучающей выборки в соответствии со значениями метрики ROUGE-1, вычисленных для текстов и заголовков, фильтрации данных и генерации искусственных обучающих примеров. Предложенный подход, протестированный на примере нейросетевой модели BART, показал улучшение качества генерации заголовков на материале двух англоязычных корпусов

Factuality Challenges in the Era of Large Language Models

The emergence of tools based on Large Language Models (LLMs), such as OpenAI's ChatGPT, Microsoft's Bing Chat, and Google's Bard, has garnered immense public attention. These incredibly useful, natural-sounding tools mark significant advances in natural language generation, yet they exhibit a propensity to generate false, erroneous, or misleading content -- commonly referred to as "hallucinations." Moreover, LLMs can be exploited for malicious applications, such as generating false but credible-sounding content and profiles at scale. This poses a significant challenge to society in terms of the potential deception of users and the increasing dissemination of inaccurate information. In light of these risks, we explore the kinds of technological innovations, regulatory reforms, and AI literacy initiatives needed from fact-checkers, news organizations, and the broader research and policy communities. By identifying the risks, the imminent threats, and some viable solutions, we seek to shed light on navigating various aspects of veracity in the era of generative AI.Comment: Our article offers a comprehensive examination of the challenges and risks associated with Large Language Models (LLMs), focusing on their potential impact on the veracity of information in today's digital landscap

Conditional Neural Headline Generation for Finnish

Automatic headline generation has the potential to significantly assist editors charged with head- lining articles. Approaches to automation in the headlining process can range from tools as creative aids, to complete end to end automation. The latter is difficult to achieve as journalistic require- ments imposed on headlines must be met with little room for error, with the requirements depending on the news brand in question. This thesis investigates automatic headline generation in the context of the Finnish newsroom. The primary question I seek to answer is how well the current state of text generation using deep neural language models can be applied to the headlining process in Finnish news media. To answer this, I have implemented and pre-trained a Finnish generative language model based on the Transformer architecture. I have fine-tuned this language model for headline generation as autoregression of headlines conditioned on the article text. I have designed and implemented a variation of the Diverse Beam Search algorithm, with additional parameters, to perform the headline generation in order to generate a diverse set of headlines for a given text. The evaluation of the generative capabilities of this system was done with real world usage in mind. I asked domain-experts in headlining to evaluate a generated set of text-headline pairs. The task was to accept or reject the individual headlines in key criteria. The responses of this survey were then quantitatively and qualitatively analyzed. Based on the analysis and feedback, this model can already be useful as a creative aid in the newsroom despite being far from ready for automation. I have identified concrete improvement directions based on the most common types of errors, and this provides interesting future work