Unleashing the Power of VGG16: Advancements in Facial Emotion Recognization

In facial emotion detection, researchers are actively exploring effective methods to identify and understand facial expressions. This study introduces a novel mechanism for emotion identification using diverse facial photos captured under varying lighting conditions. A meticulously pre-processed dataset ensures data consistency and quality. Leveraging deep learning architectures, the study utilizes feature extraction techniques to capture subtle emotive cues and build an emotion classification model using convolutional neural networks (CNNs). The proposed methodology achieves an impressive 97% accuracy on the validation set, outperforming previous methods in terms of accuracy and robustness. Challenges such as lighting variations, head posture, and occlusions are acknowledged, and multimodal approaches incorporating additional modalities like auditory or physiological data are suggested for further improvement. The outcomes of this research have wide-ranging implications for affective computing, human-computer interaction, and mental health diagnosis, advancing the field of facial emotion identification and paving the way for sophisticated technology capable of understanding and responding to human emotions across diverse domains

Facial expression recognition and intensity estimation.

Doctoral Degree. University of KwaZulu-Natal, Durban.Facial Expression is one of the profound non-verbal channels through which human emotion state is inferred from the deformation or movement of face components when facial muscles are activated. Facial Expression Recognition (FER) is one of the relevant research fields in Computer Vision (CV) and Human-Computer Interraction (HCI). Its application is not limited to: robotics, game, medical, education, security and marketing. FER consists of a wealth of information. Categorising the information into primary emotion states only limit its performance. This thesis considers investigating an approach that simultaneously predicts the emotional state of facial expression images and the corresponding degree of intensity. The task also extends to resolving FER ambiguous nature and annotation inconsistencies with a label distribution learning method that considers correlation among data. We first proposed a multi-label approach for FER and its intensity estimation using advanced machine learning techniques. According to our findings, this approach has not been considered for emotion and intensity estimation in the field before. The approach used problem transformation to present FER as a multilabel task, such that every facial expression image has unique emotion information alongside the corresponding degree of intensity at which the emotion is displayed. A Convolutional Neural Network (CNN) with a sigmoid function at the final layer is the classifier for the model. The model termed ML-CNN (Multilabel Convolutional Neural Network) successfully achieve concurrent prediction of emotion and intensity estimation. ML-CNN prediction is challenged with overfitting and intraclass and interclass variations. We employ Visual Geometric Graphics-16 (VGG-16) pretrained network to resolve the overfitting challenge and the aggregation of island loss and binary cross-entropy loss to minimise the effect of intraclass and interclass variations. The enhanced ML-CNN model shows promising results and outstanding performance than other standard multilabel algorithms. Finally, we approach data annotation inconsistency and ambiguity in FER data using isomap manifold learning with Graph Convolutional Networks (GCN). The GCN uses the distance along the isomap manifold as the edge weight, which appropriately models the similarity between adjacent nodes for emotion predictions. The proposed method produces a promising result in comparison with the state-of-the-art methods.Author's List of Publication is on page xi of this thesis

Issues in Computer Vision Data Collection: Bias, Consent, and Label Taxonomy

Recent success of the convolutional neural network in image classification has pushed the computer vision community towards data-rich methods of deep learning. As a consequence of this shift, the data collection process has had to adapt, becoming increasingly automated and efficient to satisfy algorithms that require massive amounts of data. In the push for more data, however, careful consideration into decisions and assumptions in the data collection process have been neglected. Likewise, users accept datasets and their embed- ded assumptions at face-value, employing them in theory and application papers without scrutiny. As a result, undesirable biases, non-consensual data collection, and inappropriate label taxonomies are rife in computer vision datasets. This work aims to explore issues of bias, consent, and label taxonomy in computer vision through novel investigations into widely-used datasets in image classification, face recognition, and facial expression recognition. Through this work, I aim to challenge researchers to reconsider normative data collection and use practices such that computer vision systems can be developed in a more thoughtful and responsible manner

The attachment and separation experiences of the left-behind children in rural China

The left-behind children (LBC) refer to children living in rural China, who have been left behind by parents going to work in urban regions. The population of LBC is a social by-product of labour migration in society. LBC grow up experiencing prolonged parent-child separations. Research has shown that LBC suffered from emotional distress and were at risk of reduced psycho-social development. While assessing the existing literature on the impact of the parental migration on LBC’s attachment, a meta-analysis was conducted and showed that LBC’s parental attachment security was significantly lower than that of non-left-behind children (NLBC). In the current literature, the children are often perceived as passive victims of adverse life experiences and whose own voices are neglected. Therefore, it is imperative that we understand the key processes of the LBC’s personal experiences of parental migration from their own perspectives. The current research draws on the theoretical framework of attachment theory to explore the LBC’s experiences of the parent-child attachment and the migratory separations from parents. This research used a mixed-method research design, with a quantitative study using an interview- based measure to investigate the distribution of attachment styles of the LBC, a qualitative study using the grounded theory to explore the children’s experiences and a triangulation analysis integrating the findings from the quantitative study and the qualitative study. Thirty-nine LBC participated in the study. All the children experienced lengthy separation from both their migrant parents. Findings revealed that the LBC had high rates of the dismissing attachment styles towards their fathers and mothers, similar to children experiencing other types of separation, such as children in foster care. However, findings also showed that it was possible for some LBC to maintain secure attachment with their parents despite the prolonged separations. The grounded theory focused on the LBC’s agency in doing rural migrant families. During the pre-migration stage, children were placed in a position in their families that lacked agency. They were largely ignored during the decision-making of the migration. The rural migrant families as perceived by the LBC was conceptualised as ‘doing rural migrant families’ that foregrounds the active and ongoing essential family practices of ‘building family collaboration’, ‘maintaining an intact family’ and ‘negotiating support and constraints’. Though starting with a passive position of little agency, the LBC manged to exercise both of their self- focused agency (i.e., making meaning and self-regulation) and other-focused agency (i.e., constructive compliance, resistance, support seeking and giving) in all these family practices. Self-agency was vital to the LBC’s resilience when facing parental migration, and their resilience was part of their whole rural family’s resilience. The processes in the grounded theory suggested key resilience processes in the successful adaptation to the migration for the LBC and their families. The triangulation of the results from the quantitative and qualitative studies suggested that children’s attachment styles might be associated with different parent-child communications, children’s emotional responses and cognitive perceptions. The finding of the current research had implications for the future practices and policies for the LBC and the rural migrant families. The support and care provided for the children should consider the children’s personal agencies and be designed according to the children’s own needs

Pedestrian soft biometrics recognition using deep learning on thermal images in smart cities

With technological advancement and the rise of the Internet of Things, our society is becoming more interconnected than ever before. Our computers and devices are getting smaller, and their computing power and memory has been increasing. These advances coupled with the leaps in artificial intelligence caused by the deep learning revolution in recent yearshave led to an increasingly rising interest in the field of pervasive intelligence. Intelligence in the environment has been used in smart homes in order to bring assistance to semi-autonomous people by performing activity recognition based on sensor data. As technology keeps improving, we may start to investigate the extension of assistive technologies beyond the boundaries of smart homes and into our smart cities. In order to bring assistance to semi-autonomous people, the first step is to be able to recognize profiles of vulnerable people. In order to leverage technology and artificial intelligence to make our cities smarter, safer and more accessible, this thesis investigates the use of environmental sensors such as thermal cameras to perform pedestrian soft biometrics recognition (age, gender and mobility) in the city. In this thesis, the process of building prototypes from scratch in order to collect thermal gait data in the city is explored, and the use and optimization of deep learning algorithms to perform soft biometrics recognition, as well as the feasibility of implementing these algorithms on limited resource boards are explored. The use of unprocessed thermal images allows a higher degree of privacy for the citizens, and it is novel in the field of human profile recognition. This thesis aims to set the foundation of future work, both in the field of thermal images-based soft biometrics recognition and pervasive intelligence in our cities in order to make them smarter, and move towards an interconnected society. Les progrès technologiques et le développement de l’Internet des Objets nous mènent vers une société de plus en plus interconnectée. Nos ordinateurs et nos appareils deviennent de plus en plus petits et leur puissance de calcul et leur mémoire ne cesse de s’améliorer. Ces avancées combinées aux récents progrès dans le domaine de l’intelligence artificielle avec la révolution de l’apprentissage profond ont mené à un intérêt grandissant dans le domaine de l’intelligence ambiante. L’intelligence ambiante a été utilisée dans le domaine des maisons intelligentes sous forme de reconnaissance d’activités, permettant d’assister les personnes semi-autonomes en utilisant des données collectées par des capteurs. Alors que le progrès technologique continue, nous arrivons à un point où l’hypothèse d’étendre ces stratégies d’assistance des maisons aux villes intelligentes devient de plus en plus réaliste. Afin d’étendre cette assistance aux villes, la première étape est d’identifier les personnes vulnérables, qui sont celles qui pourraient bénéficier de cette assistance. Dans le but d’utiliser la technologie pour rendre nos villes plus intelligentes, plus sûres et plus accessibles, cette thèse explore l’utilisation de capteurs environnementaux tels que des caméras thermiques pour effectuer de la reconnaissance de profils dans la ville (âge, genre et mobilité). Dans cette thèse, le processus de construction de prototypes pour récolter des données thermales dans la ville est présenté, et l’utilisation ainsi que l’optimisation d’algorithmes d’apprentissage profond pour la reconnaissance de profils est explorée. L’implémentation des algorithmes sur un système embarqué est également abordée. L’utilisation d’images thermiques garantit un plus grand degré d’anonymat pour les citoyens que l’utilisation de caméras RGB, et cette thèse représente les premiers travaux de reconnaissance de profils multiples en utilisant uniquement des images thermiques sans pré-traitement. Cette thèse a pour objectif de poser les bases pour des travaux futurs dans le domaine de la reconnaissance de profils en utilisant des images thermiques, ainsi que dans le domaine de l’intelligence ambiante dans nos villes, afin de les rendre plus intelligentes et de se diriger vers une société interconnectée

VISUAL FRAMING ON ARAB SATELLITE TV: COMPARING THE CONTENT AND STRUCTURE OF AL JAZEERA, AL JAZEERA ENGLISH, AL ARABIYA, ALHURRA, AND BBC ARABIC NEWSCASTS

Initially praised as independent voices in a region known for authoritarian control, the pan-Arab satellite news networks have been criticized for airing content that is overtly violent, and sensational. Guided by framing theory and media categorization based in press theory and political economy models, a quantitative content analysis was conducted on news programming from five networks--Al Jazeera, Al Jazeera English, Al Arabiya, Alhurra, and BBC Arabic--to determine if differences exist between the networks, and between 2 dimensions of a network taxonomy--western and liberal commercial--in how news selection, content, and sensational structural features are framed. The data from 6,595 shots and 438 stories reveal both similarities and differences in how the networks visually framed their daily newscasts and Arab Spring coverage. Differences were found in the nominal agenda diversity of stories aired between western networks emphasizing more sports, and liberal commercial networks emphasizing more teases. Alhurra presented more Amerocentric visuals than the other networks. The liberal commercial networks utilized more conflict visuals than western networks. In addition, the conflict visuals on liberal networks were more likely to depict explicit violence. The study also predicted differing applications of sensational production techniques among the networks. Use of sensational presentation features was found to be more likely on liberal networks. Al Arabiya utilized more sensational presentation features than the other networks. Visuals from Arab Spring coverage were also investigated. The western networks presented more non-violent visuals, while the liberal networks presented a more explicitly violent view of the uprisings. Al Jazeera English aired the most visuals containing conflict. Al Jazeera's coverage was the most explicitly violent. No difference was found in the application of the human interest frame between western and liberal networks. However, Alhurra invoked the political frame more than Al Jazeera and BBC Arabic. Finally, use of sensational production techniques in Arab Spring coverage mirrored routine news coverage. The liberal networks relied on more sensational presentation features. Al Arabiya's presentation style was the most sensational

Facebook, the Media and Democracy

Facebook, the Media and Democracy examines Facebook Inc. and the impact that it has had and continues to have on media and democracy around the world. Drawing on interviews with Facebook users of different kinds and dialogue with politicians, regulators, civil society and media commentators, as well as detailed documentary scrutiny of legislative and regulatory proposals and Facebook’s corporate statements, the book presents a comprehensive but clear overview of the current debate around Facebook and the global debate on the regulation of social media in the era of ‘surveillance capitalism.’ Chapters examine the business and growing institutional power of Facebook as it has unfolded over the fifteen years since its creation, the benefits and meanings that it has provided for its users, its disruptive challenge to the contemporary media environment, its shaping of conversations, and the emerging calls for its further regulation. The book considers Facebook’s alleged role in the rise of democratic movements around the world as well as its suggested role in the election of Donald Trump and the UK vote to leave the European Union. This book argues that Facebook, in some shape or form, is likely to be with us into the foreseeable future and that how we address the societal challenges that it provokes, and the economic system that underpins it, will define how human societies demonstrate their capacity to protect and enhance democracy and ensure that no corporation can set itself above democratic institutions. This is an important research volume for academics and researchers in the areas of media studies, communications, social media and political science

Personality Identification from Social Media Using Deep Learning: A Review

Social media helps in sharing of ideas and information among people scattered around the world and thus helps in creating communities, groups, and virtual networks. Identification of personality is significant in many types of applications such as in detecting the mental state or character of a person, predicting job satisfaction, professional and personal relationship success, in recommendation systems. Personality is also an important factor to determine individual variation in thoughts, feelings, and conduct systems. According to the survey of Global social media research in 2018, approximately 3.196 billion social media users are in worldwide. The numbers are estimated to grow rapidly further with the use of mobile smart devices and advancement in technology. Support vector machine (SVM), Naive Bayes (NB), Multilayer perceptron neural network, and convolutional neural network (CNN) are some of the machine learning techniques used for personality identification in the literature review. This paper presents various studies conducted in identifying the personality of social media users with the help of machine learning approaches and the recent studies that targeted to predict the personality of online social media (OSM) users are reviewed

Media strategies and coverage of international conflicts : The 2003 Iraq War and Al-Jazeera

In 2003 the United States of America led an international coalition to topple Saddam Hussein's regime in Iraq. The war on Iraq followed the war launched on Afghanistan in 2001, designed to topple the Taliban regime. In both conflicts a wide range of media strategies were implemented by the Coalition forces to sway domestic and international public opinion and to construct support for the US-led military campaigns. This research explores the media strategies implemented in the 2003 Iraq war and the policies of coverage that were used to report the conflict by the Al-Jazeera satellite channel. The major research question is to ask what developments took place in wartime media strategies during these conflicts and to investigate the way media conditions changed, especially around the rise of Al-Jazeera, and the role it played in covering the war. In order to answer these questions, it was essential to review conflicts of a similar nature, such as the 1956 Suez Canal war, the 1991 Gulf war, the 1999 Kosovo war and the 2001 war in Afghanistan. The thesis argues that the toppling of regimes was a [text unavailable] conflicts, and thus, that media strategies and techniques followed similar patterns in each case. Lessons from these conflicts had considerable impact on the 2003 Iraq war. Media strategies in this conflict were a product of lessons from previous experiences, the outcome of remarkable developments in communications technologies, and a result of the increasingly complex influence of political, economic and social factors on the way modern conflicts are mediatized. In this thesis the mediatisation of conflicts is the research thematic approach which is used to make sense of the role of these various complex factors in the production of media output. The overlapping of these factors contributes to the presentation and the perception of modern conflicts. In the case of the 2003 Iraq war, Al-Jazeera and other Arab satellite channels expanded the news agenda to include an alternative perspective to the western mainstream media. This thesis argues that this was a major development which had a critical effect on the flow of information, and radically challenged existing mainstream news management policies. Thus, studying Al-Jazeera in relation to the coverage of the 2003 Iraq war became a crucial element in understanding the changes in the way contemporary conflicts are communicated and reported, which is the central focus of this research. A triangulation of qualitative research methods has been applied to examine the issues this thesis is critically assessing. Documentary research, including on-line research, was used to explore media strategies during the 2003 Iraq war and to establish the patterns within these. The same method was applied to explore Al-Jazeera's policies of coverage. In addition, the research used in-depth interviews and an ethnographic approach, spending time for example in Al-Jazeera's newsrooms, in order to answer the main research question. This was to assess the challenges Al-Jazeera, as an Arab news provider, posed to US policies of information control and news management during the conflicts discussed above, and how, as a result, the emergence of a new mediascape in the Arab world came to challenge policy makers, media strategists and media organisations alike