PDANet: Polarity-consistent Deep Attention Network for Fine-grained Visual Emotion Regression

Existing methods on visual emotion analysis mainly focus on coarse-grained emotion classification, i.e. assigning an image with a dominant discrete emotion category. However, these methods cannot well reflect the complexity and subtlety of emotions. In this paper, we study the fine-grained regression problem of visual emotions based on convolutional neural networks (CNNs). Specifically, we develop a Polarity-consistent Deep Attention Network (PDANet), a novel network architecture that integrates attention into a CNN with an emotion polarity constraint. First, we propose to incorporate both spatial and channel-wise attentions into a CNN for visual emotion regression, which jointly considers the local spatial connectivity patterns along each channel and the interdependency between different channels. Second, we design a novel regression loss, i.e. polarity-consistent regression (PCR) loss, based on the weakly supervised emotion polarity to guide the attention generation. By optimizing the PCR loss, PDANet can generate a polarity preserved attention map and thus improve the emotion regression performance. Extensive experiments are conducted on the IAPS, NAPS, and EMOTIC datasets, and the results demonstrate that the proposed PDANet outperforms the state-of-the-art approaches by a large margin for fine-grained visual emotion regression. Our source code is released at: https://github.com/ZizhouJia/PDANet.Comment: Accepted by ACM Multimedia 201

Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation

This paper surveys the current state of the art in Natural Language Generation (NLG), defined as the task of generating text or speech from non-linguistic input. A survey of NLG is timely in view of the changes that the field has undergone over the past decade or so, especially in relation to new (usually data-driven) methods, as well as new applications of NLG technology. This survey therefore aims to (a) give an up-to-date synthesis of research on the core tasks in NLG and the architectures adopted in which such tasks are organised; (b) highlight a number of relatively recent research topics that have arisen partly as a result of growing synergies between NLG and other areas of artificial intelligence; (c) draw attention to the challenges in NLG evaluation, relating them to similar challenges faced in other areas of Natural Language Processing, with an emphasis on different evaluation methods and the relationships between them.Comment: Published in Journal of AI Research (JAIR), volume 61, pp 75-170. 118 pages, 8 figures, 1 tabl

Human Visual Perception, study and applications to understanding Images and Videos

Ph.DDOCTOR OF PHILOSOPH

Deep Multibranch Fusion Residual Network for Insect Pest Recognition

Earlier insect pest recognition is one of the critical factors for agricultural yield. Thus, an effective method to recognize the category of insect pests has become significant issues in the agricultural field. In this paper, we proposed a new residual block to learn multi-scale representation. In each block, it contains three branches: one is parameter-free, and the others contain several successive convolution layers. Moreover, we proposed a module and embedded it into the new residual block to recalibrate the channel-wise feature response and to model the relationship of the three branches. By stacking this kind of block, we constructed the Deep Multi-branch Fusion Residual Network (DMF-ResNet). For evaluating the model performance, we first test our model on CIFAR-10 and CIFAR-100 benchmark datasets. The experimental results show that DMF-ResNet outperforms the baseline models significantly. Then, we construct DMF-ResNet with different depths for high-resolution image classification tasks and apply it to recognize insect pests. We evaluate the model performance on the IP102 dataset, and the experimental results show that DMF-ResNet could achieve the best accuracy performance than the baseline models and other state-of-art methods. Based on these empirical experiments, we demonstrate the effectiveness of our approach

Compassion or neoliberal governance: critiquing the discourse of compassionate Louisville.

Since 2011, the city of Louisville, the site of this research, has been calling itself “Compassionate City.” The dissertation takes Compassionate Louisville as a city brand and seeks to investigate the effect of the image construction on the local politics. It aims to investigate the latent discourses produced by Compassionate Louisville in relation to the neoliberal political economy of the city. This dissertation employs the Critical Discourse Analysis (CDA) methodology. Compassionate Louisville, being a brand, produces a discourse of compassion. The CDA methodological approach helps to investigate this manufactured discourse in relation to the neoliberal socio-political context of the city. To conduct the study, the dissertation has analyzed textual data from a variety of sources, including interviews of key participants, city government reports, planning documents, meeting minutes of Metro Council, local news publications, websites of the city government, and relevant non-profit organizations. There are three specific but interrelated findings of this dissertation. First, the discourse produced by Compassionate Louisville permeates the political narrative of Louisville and is increasingly used in policy rationales, contestation, debates, and claim-making. The discourse is strategically used by various groups, including politicians, city officials, religious organizations, activists, non-profits, and businesses. Second, the discourse of compassion aids neoliberalism by privatizing the responsibility of welfare in the disguise of a moral high ground of compassion. In the process, it depoliticizes social problems, displaces rights, and entrenches the precarity of the communities. Third, the discourse of compassion is an urban version of humanitarian governance and acts as a technology of neoliberalism. It serves to manage the marginalized population of the city, discipline emotions to make the working population more compliant, and create the ground to transform emotions into a productive asset through which value can be extracted. Taken together, the dissertation finds that the discourse of compassion in Louisville originates and operates in the social context of neoliberalism- it works the work of capital in the disguise of a humanitarian narrative

Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Work

Inspired by the fact that human brains can emphasize discriminative parts of the input and suppress irrelevant ones, substantial local mechanisms have been designed to boost the development of computer vision. They can not only focus on target parts to learn discriminative local representations, but also process information selectively to improve the efficiency. In terms of application scenarios and paradigms, local mechanisms have different characteristics. In this survey, we provide a systematic review of local mechanisms for various computer vision tasks and approaches, including fine-grained visual recognition, person re-identification, few-/zero-shot learning, multi-modal learning, self-supervised learning, Vision Transformers, and so on. Categorization of local mechanisms in each field is summarized. Then, advantages and disadvantages for every category are analyzed deeply, leaving room for exploration. Finally, future research directions about local mechanisms have also been discussed that may benefit future works. To the best our knowledge, this is the first survey about local mechanisms on computer vision. We hope that this survey can shed light on future research in the computer vision field

An Investigation Of The Relationship Between The Use Of Modern Digital Technologies, Language Learning Strategies, And Development Of Second Language Skills

Like many other areas of human knowledge, the field of language learning has undergone changes as a consequence of the application of digital technologies. Extensive exposure and anytime and anywhere access availability to data in a second or foreign language (L2) bring almost unlimited learning opportunities for digital age students, which affects their learning behaviors also known as language learning strategies (LLS). The purpose of the present study is to define preferred LLS patterns of digitally native L2 learners and to establish relationships between types of existing digital technologies, learners’ demographic characteristics, and the use of learning strategies to support the development of specific language skills and aspects. The setting for this study was made up by a medium-sized university in the northern U.S., particularly, its undergraduate student population enrolled in foreign language courses in the Department of Modern and Classical Languages and Literatures during the 2021 fall semester. They were asked to complete a survey that contained the original validated version of the Strategy Inventory for Language Learning (SILL) instrument (Oxford, 1990) and three additional sections disclosing the participants’ demographics, technology use experience, and targeted language skills and aspects. Both descriptive and inferential quantitative methods of data analysis were used in the study to elucidate the research questions. A number of analytic procedures using SPSS® Statistics software were performed to find out detailed statistic values of the research variables. Frequencies and descriptive statistics, analysis of correlations, extreme groupings t-tests to explore the relationships between the subsets of categorical variables, and factor analysis of LLS domains were implemented to identify meaningful patterns of technology use in L2 learning. Data from this study provide a view of how the Digital Natives themselves see their technology use and approaches to learning. Research conclusions based on obtained self-reported evidence allow us to make broader recommendations for changes in the L2 teaching methodology. They may also prevent instructors from making unsupported assumptions about their students\u27 mastery of educational technology, and, thereby, from neglecting to teach students the skills they need for academic success. Keywords: digital native learner, digital technology categories, language learning strategies, L2 language skill

Bringing Instagram Posts into Being: A Study of FYC Students\u27 Self-Sponsored Posting Practices and Transfer Opportunities

Social media platforms have offered students—and all of us—more opportunities for self-sponsored writing. In response to calls from researchers to explore students\u27 21st-century writing practices and their relevance to college writing instruction, this dissertation articulated and applied a feminist teacher research methodology and a mixed-methods research design to explore first-year composition (FYC) students\u27 self-sponsored writing practices, attitudes, and transfer opportunities on a popular, albeit under-examined, social media application: Instagram. This study found that students have developed elaborate, rhetorical, multimodal composing processes that include planning, drafting, evaluating, selecting, and styling images as well as planning, drafting/revising, and styling captions. Additionally, though most survey participants said that audience awareness figured into their composing practices, data from interviews revealed that students often misunderstood or inaccurately specified their audiences. Similarly, while all interviewees used a process-based approach to compose their Instagram posts, significant differences exist regarding students\u27 levels of awareness about their composing decisions. Concerning students\u27 perceptions of transfer opportunities between Instagram and FYC, this study found that most survey respondents did not conceptualize their Instagram writing as writing nor did they see their Instagram writing practices as related to the writing required in FYC. Further, respondents generally disagreed that opportunities to transfer skills and knowledge learned from Instagram to FYC exist. However, student interviewees offered evidence that contradicted survey results. Specifically, all interviewees within the study cited connections between their writing practices on Instagram and FYC composing practices by the end of their interviews. Findings from this study productively extend and nuance prior research on students\u27 extracurricular composing practices, offer new findings that address the lack of empirical data about Instagram and writing process, and have several implications for FYC pedagogy. Particular curricular suggestions are provided along with two guiding principles that extend this dissertation\u27s results