Investigating effects of emoji on neutral narrative text: evidence from eye movements and perceived emotional valence

Digital images of faces such as emoji in virtual communication have become increasingly popular, but current research findings are inconsistent regarding their emotional effects on perceptions of text. Similarly, emoji effects on reading behaviours are largely unknown and require further examination. The present study (N = 41) investigated how the position and emotional valence of emoji in neutral narrative sentences influenced eye movements during reading and perceptions of sentence valence. Participants read neutral narrative sentences containing smiling or frowning emoji in sentence-initial or sentence-final positions and rated the perceived emotional valence of the sentence. Results from linear mixed-effects models demonstrated significantly longer fixations on sentence-final emoji and longer sentence reading times when emoji were in sentence-final positions. These findings are comparable to sentence ‘wrap-up’ effects witnessed in the processing of lexical units during sentence reading, providing new evidence towards the way readers integrate emoji into contextual processing. However, no impact of emoji valence or position on first-pass target word processing or sentence-valence ratings were found. This would refute previous suggestions that digital faces influence text valence, raising questions about reader preference for emoji or sentence sentiment, the influence of sentence formatting, and delivery/display mechanism on these effects

Interactions between text content and emoji types determine perceptions of both messages and senders

Emoji increasingly feature alongside written language in interpersonal communication. Boutet et al. (2021) showed that negative-face emoji led to a negativity effect on perceptions of message tone and senders' mood. We extended their design, considering the role of non-face emoji and the impact of text content and emoji on message clarity. We utilised a 3 (sentence valence: negative, neutral, positive) × 5 (emoji type: no emoji, negative face, neutral face, positive face, object emoji) quasi-experimental design and online survey method. Sixty participants each processed 60 stimuli counterbalanced across conditions, rating messages' emotional tone and clarity, and senders’ warmth and emotional state. Cumulative link mixed models were used to analyse responses. We found that sentence valence and emoji type interact, influencing message emotionality and clarity, and perceived sender warmth and state. The congruency of text and emoji was particularly important; results showed that incongruent emoji detracted from message clarity vs. no emoji (or congruent emoji). Congruent emoji typically amplified emotional perceptions of messages and senders. Object emoji were most influential when text was either neutral or positive. Results were consistent with models such as the EASI framework (Van Kleef, 2009), and suggest that compositionality extends to representations of text + emoji

Emoji alter the perception of emotion in affectively neutral text messages

Previous studies of emoji effects on text sentiment demonstrate mixed findings. Further, these studies are limited by confounds, e.g., underlying text sentiment, lack of ecological validity. We considered emoji effects on the emotional valence of affectively neutral English-language text messages. We additionally considered differences across US-American, British, and Danish participants. 217 participants considered screenshots of question-and-response text message exchanges with/without emoji, in a 4 (emoji type: no emoji, negative, neutral, positive) × 3 (nationality: American, British, Danish) mixed-factors design. Cumulative link mixed-effects models demonstrated that messages + negative emoji were rated more-negatively than any other emoji conditions. Responses + positive emoji were rated more-positively than any other emoji condition. Responses + neutral emoji and responses without emoji were perceived as equally emotive. There was no emoji type × nationality interaction, suggesting that emoji effects were consistent across participants. Findings are considered viz linguistic processing, social interactions, education, marketing, and public health interventions

Emoji identification and emoji effects on sentence emotionality in ASD-diagnosed adults and neurotypical controls

We investigated ASD-diagnosed adults’ and neurotypical (NT) controls’ processing of emoji and emoji influence on the emotionality of otherwise-neutral sentences. Study 1 participants categorised emoji representing the six basic emotions using a fixed-set of emotional adjectives. Results showed that ASD-diagnosed participants’ classifications of fearful, sad, and surprised emoji were more diverse and less ‘typical’ than NT controls’ responses. Study 2 participants read emotionally-neutral sentences; half paired with sentence-final happy emoji, half with sad emoji. Participants rated sentence + emoji stimuli for emotional valence. ASD-diagnosed and NT participants rated sentences + happy emoji as equally-positive, however, ASD-diagnosed participants rated sentences + sad emoji as more-negative than NT participants. We must acknowledge differential perceptions and effects of emoji, and emoji-text inter-relationships, when working with neurodiverse stakeholders