Learning Emotion Representations from Verbal and Nonverbal Communication

Emotion understanding is an essential but highly challenging component of artificial general intelligence. The absence of extensively annotated datasets has significantly impeded advancements in this field. We present EmotionCLIP, the first pre-training paradigm to extract visual emotion representations from verbal and nonverbal communication using only uncurated data. Compared to numerical labels or descriptions used in previous methods, communication naturally contains emotion information. Furthermore, acquiring emotion representations from communication is more congruent with the human learning process. We guide EmotionCLIP to attend to nonverbal emotion cues through subject-aware context encoding and verbal emotion cues using sentiment-guided contrastive learning. Extensive experiments validate the effectiveness and transferability of EmotionCLIP. Using merely linear-probe evaluation protocol, EmotionCLIP outperforms the state-of-the-art supervised visual emotion recognition methods and rivals many multimodal approaches across various benchmarks. We anticipate that the advent of EmotionCLIP will address the prevailing issue of data scarcity in emotion understanding, thereby fostering progress in related domains. The code and pre-trained models are available at https://github.com/Xeaver/EmotionCLIP.Comment: CVPR 202

Furnishing the Galaxy with Pulsars

The majority of pulsar population synthesis studies performed to date have focused on isolated pulsar evolution. Those that have incorporated pulsar evolution within binary systems have tended to either treat binary evolution poorly of evolve the pulsar population in an ad-hoc manner. Here we present the first model of the Galactic field pulsar population that includes a comprehensive treatment of both binary and pulsar evolution. Synthetic observational surveys mimicking a variety of radio telescopes are then performed on this population. As such, a complete and direct comparison of model data with observations of the pulsar population within the Galactic disk is now possible. The tool used for completing this work is a code comprised of three components: stellar/binary evolution, Galactic kinematics and survey selection effects. Here we give a brief overview of the method and assumptions involved with each component. Some preliminary results are also presented as well as plans for future applications of the code.Comment: 3 pages, 3 figures, Conference: "40 years of pulsars: Millisecond pulsars, magnetars and more", McGill University, Montreal, Canada, ed. A.Cumming et al., AI