Exploring Fairness in Pre-trained Visual Transformer based Natural and GAN Generated Image Detection Systems and Understanding the Impact of Image Compression in Fairness

It is not only sufficient to construct computational models that can accurately classify or detect fake images from real images taken from a camera, but it is also important to ensure whether these computational models are fair enough or produce biased outcomes that can eventually harm certain social groups or cause serious security threats. Exploring fairness in forensic algorithms is an initial step towards correcting these biases. Since visual transformers are recently being widely used in most image classification based tasks due to their capability to produce high accuracies, this study tries to explore bias in the transformer based image forensic algorithms that classify natural and GAN generated images. By procuring a bias evaluation corpora, this study analyzes bias in gender, racial, affective, and intersectional domains using a wide set of individual and pairwise bias evaluation measures. As the generalizability of the algorithms against image compression is an important factor to be considered in forensic tasks, this study also analyzes the role of image compression on model bias. Hence to study the impact of image compression on model bias, a two phase evaluation setting is followed, where a set of experiments is carried out in the uncompressed evaluation setting and the other in the compressed evaluation setting

A Robust Approach Towards Distinguishing Natural and Computer Generated Images using Multi-Colorspace fused and Enriched Vision Transformer

The works in literature classifying natural and computer generated images are mostly designed as binary tasks either considering natural images versus computer graphics images only or natural images versus GAN generated images only, but not natural images versus both classes of the generated images. Also, even though this forensic classification task of distinguishing natural and computer generated images gets the support of the new convolutional neural networks and transformer based architectures that can give remarkable classification accuracies, they are seen to fail over the images that have undergone some post-processing operations usually performed to deceive the forensic algorithms, such as JPEG compression, gaussian noise, etc. This work proposes a robust approach towards distinguishing natural and computer generated images including both, computer graphics and GAN generated images using a fusion of two vision transformers where each of the transformer networks operates in different color spaces, one in RGB and the other in YCbCr color space. The proposed approach achieves high performance gain when compared to a set of baselines, and also achieves higher robustness and generalizability than the baselines. The features of the proposed model when visualized are seen to obtain higher separability for the classes than the input image features and the baseline features. This work also studies the attention map visualizations of the networks of the fused model and observes that the proposed methodology can capture more image information relevant to the forensic task of classifying natural and generated images

Blacks is to Anger as Whites is to Joy? Understanding Latent Affective Bias in Large Pre-trained Neural Language Models

Groundbreaking inventions and highly significant performance improvements in deep learning based Natural Language Processing are witnessed through the development of transformer based large Pre-trained Language Models (PLMs). The wide availability of unlabeled data within human generated data deluge along with self-supervised learning strategy helps to accelerate the success of large PLMs in language generation, language understanding, etc. But at the same time, latent historical bias/unfairness in human minds towards a particular gender, race, etc., encoded unintentionally/intentionally into the corpora harms and questions the utility and efficacy of large PLMs in many real-world applications, particularly for the protected groups. In this paper, we present an extensive investigation towards understanding the existence of "Affective Bias" in large PLMs to unveil any biased association of emotions such as anger, fear, joy, etc., towards a particular gender, race or religion with respect to the downstream task of textual emotion detection. We conduct our exploration of affective bias from the very initial stage of corpus level affective bias analysis by searching for imbalanced distribution of affective words within a domain, in large scale corpora that are used to pre-train and fine-tune PLMs. Later, to quantify affective bias in model predictions, we perform an extensive set of class-based and intensity-based evaluations using various bias evaluation corpora. Our results show the existence of statistically significant affective bias in the PLM based emotion detection systems, indicating biased association of certain emotions towards a particular gender, race, and religion

REDAffectiveLM: Leveraging Affect Enriched Embedding and Transformer-based Neural Language Model for Readers' Emotion Detection

Technological advancements in web platforms allow people to express and share emotions towards textual write-ups written and shared by others. This brings about different interesting domains for analysis; emotion expressed by the writer and emotion elicited from the readers. In this paper, we propose a novel approach for Readers' Emotion Detection from short-text documents using a deep learning model called REDAffectiveLM. Within state-of-the-art NLP tasks, it is well understood that utilizing context-specific representations from transformer-based pre-trained language models helps achieve improved performance. Within this affective computing task, we explore how incorporating affective information can further enhance performance. Towards this, we leverage context-specific and affect enriched representations by using a transformer-based pre-trained language model in tandem with affect enriched Bi-LSTM+Attention. For empirical evaluation, we procure a new dataset REN-20k, besides using RENh-4k and SemEval-2007. We evaluate the performance of our REDAffectiveLM rigorously across these datasets, against a vast set of state-of-the-art baselines, where our model consistently outperforms baselines and obtains statistically significant results. Our results establish that utilizing affect enriched representation along with context-specific representation within a neural architecture can considerably enhance readers' emotion detection. Since the impact of affect enrichment specifically in readers' emotion detection isn't well explored, we conduct a detailed analysis over affect enriched Bi-LSTM+Attention using qualitative and quantitative model behavior evaluation techniques. We observe that compared to conventional semantic embedding, affect enriched embedding increases ability of the network to effectively identify and assign weightage to key terms responsible for readers' emotion detection

Affect-oriented fake news detection using machine learning

Among all other media platforms, online social media plays an important role in sharing news and information along with user opinion. This quick propagation and accumulation of information form a data deluge where it is very hard to believe all the pieces of information eventhough it appears to be very realistic

Cross-domain sentiment analysis on social media interactions using senti-lexicon based hybrid features

Analyzing the sentiment information from the social media interactions is a rapidly growing research area. Several studies in the literature focus on modeling the sentiment information using linguistics, generic word counts and even the contextual information, including the presence of punctuations, elongated words, emoticons, etc. In this paper, we experiment on the effectiveness of lexicon information in combination with other information, for the effective analysis of sentiment in social interactions. The objective of this study is to experimentally verify how senti-lexicons can take part in the process of modeling the sentiment information even in cross-domain sentiment analysis. In general, this paper explores the effectiveness of several feature vectors including the generic Bag of Word (BoW), linguistic (N-Gram and Part-of-Speech (POS)) and the lexicon features (number of positive and negative words). Other than the traditional features we generate hybrid features by combining the lexicon features with the BoW and linguistic features. We conduct the experiments on sentiment classification using supervised models like Linear SVC (L-SVC), Multi-Layer Perceptron (MLP), Multinomial Naïve Bayes (MNB) and Decision Tree (DT). The experiments are conducted on three different types of sentiment document datasets - the Amazon food review dataset, student opinion tweet dataset, and the Large Movie Review Dataset v1.0. We also verify the efficacy of these features in cross-domain sentiment analysis. Experiments show that hybridizing the BoW, linguistic N-Gram and POS method with lexicon features improves the accuracy of sentiment classification even for cross-domain sentiment analysis

Indexing and retrieval of Malayalam news videos based on word image matching

News videos store a huge amount of information and are a source of historical archives. The amount of news data is growing rapidly and unpredictably, hence video indexing on news videos is a tedious job. Manual indexing even though effective, it is slow and most expensive for a massive volume of data. Content Based Indexing and Retrieval (CBIR) is a solution for this problem. Textual modality based on ticker texts is powerful enough to represent a news video since it highlights all the topics in a news bulletin. Searching and retrieval from Malayalam news videos are challenging due to the lack of effective tools for automatic content based indexing and retrieval from massive database analyzing the semantics of the news videos. The ticker texts are extracted automatically using mathematical morphology and region clustering and indexing and retrieval based on text or word image matching is implemented. Different methods like Dynamic Time Warping (DTW), Exclusive-OR (XOR), and Correlation are performed for word image matching. The features Discrete Cosine Transform (DCT) and Normalized Vertical Projection Profile (nvpp) are found to give better results

Mathematical morphology and region clustering based text information extraction from Malayalam news videos

Innovations in technologies like improved internet data transfer, advanced digital data compression algorithms, enhancements in web technology, etc. enabled the exponential growth in digital multimedia data. Among the massive multimedia data, news videos are of higher priority due to its rich up-to-date information and historical evidences. This data is rapidly growing in an unpredictable fashion which requires an efficient and powerful method to index and retrieve such massive data. Even though manual indexing is the most effective, it is the slowest and most expensive. Hence automatic video indexing is considered as an important research problem to be addressed uniquely. In this work, we propose a Mathematical Morphology and Region Clustering based Text Information Extraction (TIE) from Malayalam news videos for Content Based Video Indexing and Retrieval (CBVIR). Morphological gradient acts as an edge detector, by enhancing the intensity variations for detecting the text regions. Further an agglomerative clustering is performed to select the significant text regions. The precision, recall and F1-measure obtained for the proposed approach are 87.45%, 94.85% and 0.91 respectively

Temperature prediction using machine learning approaches

Weather prediction is one of the most important research areas due to its applicability in real-world problems like meteorology, agricultural studies, etc. We propose a method for temperature prediction using three machine learning models - Multiple Linear Regression (MLR), Artificial Neural Network (ANN) and Support Vector Machine (SVM), through a comparative analysis using the weather data collected from Central Kerala during the period 2007 to 2015. The experimental results are evaluated using Mean Error (ME), Mean Absolute Error (MAE), Root Mean Square Error (RMSE) and Correlation Coefficients (CC). The error metrics and the CC shows that MLR is a more precise model for temperature prediction than ANN and SVM