Behavioral state detection of newborns based on facial expression analysis

Prematurely born infants are observed at a Neonatal Intensive Care Unit (NICU) for medical treatment. Whereas vital body functions are continuously monitored, their incubator is covered by a blanket for medical reasons. This prevents visual observation of the newborns during most time of the day, while it is known that the facial expression can give valuable information about the presence of discomfort. This prompted the authors to develop a prototype of an automated video survey system for the detection of discomfort in newborn babies by analysis of their facial expression. Since only a reliable and situation-independent system is useful, we focus at robustness against non-ideal viewpoints and lighting conditions. Our proposed algorithm automatically segments the face from the background and localizes the eye, eyebrow and mouth regions. Based upon measurements in these regions, a hierarchical classifier is employed to discriminate between the behavioral states sleep, awake and cry. We have evaluated the described prototype system on recordings of three healthy newborns, and we show that our algorithm operates with approximately 95% accuracy. Small changes in viewpoint and lighting conditions are allowed, but when there is a major reduction in light, or when the viewpoint is far from frontal, the algorithm fails. © 2009 Springer Berlin Heidelberg

Adult judgments of infant expressive behavior: Facial, vocal, and body actions.

Adult judges were presented with videotape segments showing an infant displaying facial configurations hypothesized to express discomfort/pain, anger, or sadness according to differential emotions theory (Izard, Dougherty, & Hembree, 1983). The segments also included the infant's nonfacial behavior and aspects of the situational context. Judges rated the segments using a set of emotion terms or a set of activity terms. Results showed that judges perceived the discomfort/pain and anger segments as involving one or more negative emotions not predicted by differential emotions theory. The sadness segments were perceived as involving relatively little emotion overall. Body activity accompanying the discomfort/pain and anger configurations was judged to be more jerky and active than body activity accompanying the sadness configurations. The sadness segments were accompanied by relatively little body movement overall. The results thus fail to conform to the predictions of differential emotions theory but provide information that may contribute to the development of a theory of infant expressive behavior

Emotion processing in infancy: specificity in risk for social anxiety and associations with two year outcomes

The current study examined the specificity of patterns of responding to high and low intensity negative emotional expressions of infants of mothers with social phobia, and their association with child outcomes at two years of age. Infants of mothers with social phobia, generalised anxiety disorder (GAD) or no history of anxiety were shown pairs of angry and fearful emotional expressions at 10 weeks of age. Symptoms of social withdrawal, anxiety and sleep problems were assessed at two years of age. Only infants of mothers with social phobia showed a tendency to look away from high intensity fear faces; however infants of mothers with both social phobia and GAD showed a bias towards high intensity angry faces. Among the offspring of mothers with social phobia, anxiety symptoms at two years of age were associated with a preference for high intensity fear faces in infancy. The reverse pattern was found amongst the offspring of non-anxious mothers. These findings suggest a possible specific response to emotional expressions among the children of mothers with social phobia

Changing the game:exploring infants' participation in early play routines

Play has proved to have a central role in children’s development, most notably in rule learning (Piaget, 1965; Sutton-Smith, 1979) and negotiation of roles and goals (Garvey, 1972; Bruner et al., 1976). Yet very little research has been done on early play. The present study focuses on early social games, i.e. vocal-kinetic play routines that mothers use to interact with infants from very early on. We explored 3-month-old infants and their mothers performing a routine game first in the usual way, then in two violated conditions: without gestures and without sound. The aim of the study is to investigate infants’ participation and expectations in the game and whether this participation is affected by changes in the multimodal format of the game. Infants’ facial expressions, gaze and body movements were coded to measure levels of engagement and affective state across the three conditions. Results showed a significant decrease in Limbs Movements and expressions of Positive Affect, an increase in Gaze Away and in Stunned Expression when the game structure was violated. These results indicate that the violated game conditions were experienced as less engaging, either because of an unexpected break in the established joint routine, or simply because they were weaker versions of the same game. Overall, our results suggest that structured, multimodal play routines may constitute interactional contexts that only work as integrated units of auditory and motor resources, representing early communicative contexts which prepare the ground for later, more complex multimodal interactions, such as verbal exchanges

Eliciting Facial Expressions in Children with and without Down Syndrome

This study investigated facial expressions of children between the ages of 10 and 15 years with Down Syndrome (experimental group) and compared them to typically developing children (control group). Elicitation of facial expressions was carried out through showings of video clips. Trained observers were used to code expressions of happiness, anger, and disgust from video recordings that were made of the children’s reactions while they watched the video clips. I hypothesized that Down Syndrome children will not differ from typically developing children in the frequency of elicited happy expressions. However, I expected them to differ in the frequency of elicited anger and disgust expressions. Results support my hypotheses, showing a significant difference between the control and experimental group when comparing frequency of anger and disgust expressions, but did not differ significantly when the groups were compared in their frequency of happy expressions