Improving sentiment analysis on PeduliLindungi comments: a comparative study with CNN-Word2Vec and integrated negation handling

This study investigates sentiment analysis in Google Play reviews of the PeduliLindungi application, focusing on the integration of negation handling into text preprocessing and comparing the effectiveness of two prominent methods: CNN-Word2Vec CBOW and CNN-Word2Vec SkipGram. Through a meticulous methodology, negation handling is incorporated into the preprocessing phase to enhance sentiment analysis. The results demonstrate a noteworthy improvement in accuracy for both methods with the inclusion of negation handling, with CNN-Word2Vec SkipGram emerging as the superior performer, achieving an impressive 76.2% accuracy rate. Leveraging a dataset comprising 13,567 comments, this research introduces a novel approach by emphasizing the significance of negation handling in sentiment analysis. The study not only contributes valuable insights into the optimization of sentiment analysis processes but also provides practical considerations for refining methodologies, particularly in the context of mobile application reviews

Pretext Tasks selection for multitask self-supervised speech representation learning

Through solving pretext tasks, self-supervised learning leverages unlabeled data to extract useful latent representations replacing traditional input features in the downstream task. In audio/speech signal processing, a wide range of features where engineered through decades of research efforts. As it turns out, learning to predict such features (a.k.a pseudo-labels) has proven to be a particularly relevant pretext task, leading to useful self-supervised representations which prove to be effective for downstream tasks. However, methods and common practices for combining such pretext tasks for better performance on the downstream task have not been explored and understood properly. In fact, the process relies almost exclusively on a computationally heavy experimental procedure, which becomes intractable with the increase of the number of pretext tasks. This paper introduces a method to select a group of pretext tasks among a set of candidates. The method we propose estimates calibrated weights for the partial losses corresponding to the considered pretext tasks during the self-supervised training process. The experiments conducted on automatic speech recognition, speaker and emotion recognition validate our approach, as the groups selected and weighted with our method perform better than classic baselines, thus facilitating the selection and combination of relevant pseudo-labels for self-supervised representation learning

DeBERTNeXT: A Multimodal Fake News Detection Framework

There is a rapid influx of fake news nowadays, which poses an immense threat to our society. Fake news has been impacting us in several ways which include changing our thoughts, manipulating opinions, and also causing chaos due to misinformation. With the ease of access and sharing information on social media platforms, such fake news or misinformation has been spreading in different modalities which include text, image, audio, and video. Although there have been a lot of approaches to detecting fake news in textual format only, however, multimodal approaches are less frequent as it is difficult to fully use the information derived from different modalities to achieve high accuracy in a combined format. To tackle these issues, we introduce DeBertNeXT which is a multimodal fake news detection model that utilizes both textual and visual information from an article for fake news classification. We perform experiments on the immense Fakeddit dataset and two other smaller benchmark datasets named Politifact and Gossipcop. Our model outperforms the existing models on the Fakeddit dataset by about 3.80%, Politifact by 2.10% and Gossipcop by 1.00%

Exploring the relationship between response time sequence in scale answering process and severity of insomnia: a machine learning approach

Objectives: The study aims to investigate the relationship between insomnia and response time. Additionally, it aims to develop a machine learning model to predict the presence of insomnia in participants using response time data. Methods: A mobile application was designed to administer scale tests and collect response time data from 2729 participants. The relationship between symptom severity and response time was explored, and a machine learning model was developed to predict the presence of insomnia. Results: The result revealed a statistically significant difference (p<.001) in the total response time between participants with or without insomnia symptoms. A correlation was observed between the severity of specific insomnia aspects and response times at the individual questions level. The machine learning model demonstrated a high predictive accuracy of 0.743 in predicting insomnia symptoms based on response time data. Conclusions: These findings highlight the potential utility of response time data to evaluate cognitive and psychological measures, demonstrating the effectiveness of using response time as a diagnostic tool in the assessment of insomnia

Text-based Sentiment Analysis and Music Emotion Recognition

Nowadays, with the expansion of social media, large amounts of user-generated texts like tweets, blog posts or product reviews are shared online. Sentiment polarity analysis of such texts has become highly attractive and is utilized in recommender systems, market predictions, business intelligence and more. We also witness deep learning techniques becoming top performers on those types of tasks. There are however several problems that need to be solved for efficient use of deep neural networks on text mining and text polarity analysis. First of all, deep neural networks are data hungry. They need to be fed with datasets that are big in size, cleaned and preprocessed as well as properly labeled. Second, the modern natural language processing concept of word embeddings as a dense and distributed text feature representation solves sparsity and dimensionality problems of the traditional bag-of-words model. Still, there are various uncertainties regarding the use of word vectors: should they be generated from the same dataset that is used to train the model or it is better to source them from big and popular collections that work as generic text feature representations? Third, it is not easy for practitioners to find a simple and highly effective deep learning setup for various document lengths and types. Recurrent neural networks are weak with longer texts and optimal convolution-pooling combinations are not easily conceived. It is thus convenient to have generic neural network architectures that are effective and can adapt to various texts, encapsulating much of design complexity. This thesis addresses the above problems to provide methodological and practical insights for utilizing neural networks on sentiment analysis of texts and achieving state of the art results. Regarding the first problem, the effectiveness of various crowdsourcing alternatives is explored and two medium-sized and emotion-labeled song datasets are created utilizing social tags. One of the research interests of Telecom Italia was the exploration of relations between music emotional stimulation and driving style. Consequently, a context-aware music recommender system that aims to enhance driving comfort and safety was also designed. To address the second problem, a series of experiments with large text collections of various contents and domains were conducted. Word embeddings of different parameters were exercised and results revealed that their quality is influenced (mostly but not only) by the size of texts they were created from. When working with small text datasets, it is thus important to source word features from popular and generic word embedding collections. Regarding the third problem, a series of experiments involving convolutional and max-pooling neural layers were conducted. Various patterns relating text properties and network parameters with optimal classification accuracy were observed. Combining convolutions of words, bigrams, and trigrams with regional max-pooling layers in a couple of stacks produced the best results. The derived architecture achieves competitive performance on sentiment polarity analysis of movie, business and product reviews. Given that labeled data are becoming the bottleneck of the current deep learning systems, a future research direction could be the exploration of various data programming possibilities for constructing even bigger labeled datasets. Investigation of feature-level or decision-level ensemble techniques in the context of deep neural networks could also be fruitful. Different feature types do usually represent complementary characteristics of data. Combining word embedding and traditional text features or utilizing recurrent networks on document splits and then aggregating the predictions could further increase prediction accuracy of such models

Clustering of Distributed Word Representations and its Applicability for Enterprise Search

Machine learning of distributed word representations with neural embeddings is a state-of-the-art approach to modelling semantic relationships hidden in natural language. The thesis “Clustering of Distributed Word Representations and its Applicability for Enterprise Search” covers different aspects of how such a model can be applied to knowledge management in enterprises. A review of distributed word representations and related language modelling techniques, combined with an overview of applicable clustering algorithms, constitutes the basis for practical studies. The latter have two goals: firstly, they examine the quality of German embedding models trained with gensim and a selected choice of parameter configurations. Secondly, clusterings conducted on the resulting word representations are evaluated against the objective of retrieving immediate semantic relations for a given term. The application of the final results to company-wide knowledge management is subsequently outlined by the example of the platform intergator and conceptual extensions.":1 Introduction 1.1 Motivation 1.2 Thesis Structure 2 Related Work 3 Distributed Word Representations 3.1 History 3.2 Parallels to Biological Neurons 3.3 Feedforward and Recurrent Neural Networks 3.4 Learning Representations via Backpropagation and Stochastic Gradient Descent 3.5 Word2Vec 3.5.1 Neural Network Architectures and Update Frequency 3.5.2 Hierarchical Softmax 3.5.3 Negative Sampling 3.5.4 Parallelisation 3.5.5 Exploration of Linguistic Regularities 4 Clustering Techniques 4.1 Categorisation 4.2 The Curse of Dimensionality 5 Training and Evaluation of Neural Embedding Models 5.1 Technical Setup 5.2 Model Training 5.2.1 Corpus 5.2.2 Data Segmentation and Ordering 5.2.3 Stopword Removal 5.2.4 Morphological Reduction 5.2.5 Extraction of Multi-Word Concepts 5.2.6 Parameter Selection 5.3 Evaluation Datasets 5.3.1 Measurement Quality Concerns 5.3.2 Semantic Similarities 5.3.3 Regularities Expressed by Analogies 5.3.4 Construction of a Representative Test Set for Evaluation of Paradigmatic Relations 5.3.5 Metrics 5.4 Discussion 6 Evaluation of Semantic Clustering on Word Embeddings 6.1 Qualitative Evaluation 6.2 Discussion 6.3 Summary 7 Conceptual Integration with an Enterprise Search Platform 7.1 The intergator Search Platform 7.2 Deployment Concepts of Distributed Word Representations 7.2.1 Improved Document Retrieval 7.2.2 Improved Query Suggestions 7.2.3 Additional Support in Explorative Search 8 Conclusion 8.1 Summary 8.2 Further Work Bibliography List of Figures List of Tables Appendi

Training-Free Layout Control with Cross-Attention Guidance

Recent diffusion-based generators can produce high-quality images from textual prompts. However, they often disregard textual instructions that specify the spatial layout of the composition. We propose a simple approach that achieves robust layout control without the need for training or fine-tuning of the image generator. Our technique manipulates the cross-attention layers that the model uses to interface textual and visual information and steers the generation in the desired direction given, e.g., a user-specified layout. To determine how to best guide attention, we study the role of attention maps and explore two alternative strategies, forward and backward guidance. We thoroughly evaluate our approach on three benchmarks and provide several qualitative examples and a comparative analysis of the two strategies that demonstrate the superiority of backward guidance compared to forward guidance, as well as prior work. We further demonstrate the versatility of layout guidance by extending it to applications such as editing the layout and context of real images.Comment: WACV 2024, Project Page: https://silent-chen.github.io/layout-guidance

Text Classification: A Review, Empirical, and Experimental Evaluation

The explosive and widespread growth of data necessitates the use of text classification to extract crucial information from vast amounts of data. Consequently, there has been a surge of research in both classical and deep learning text classification methods. Despite the numerous methods proposed in the literature, there is still a pressing need for a comprehensive and up-to-date survey. Existing survey papers categorize algorithms for text classification into broad classes, which can lead to the misclassification of unrelated algorithms and incorrect assessments of their qualities and behaviors using the same metrics. To address these limitations, our paper introduces a novel methodological taxonomy that classifies algorithms hierarchically into fine-grained classes and specific techniques. The taxonomy includes methodology categories, methodology techniques, and methodology sub-techniques. Our study is the first survey to utilize this methodological taxonomy for classifying algorithms for text classification. Furthermore, our study also conducts empirical evaluation and experimental comparisons and rankings of different algorithms that employ the same specific sub-technique, different sub-techniques within the same technique, different techniques within the same category, and categorie