From Face Perception to Individual Recognition: The Missing Link

Recognizing other individuals is a key social aspect of our everyday lives. To recognize a familiar individual, we must establish a link between sensory inputs and a representation of that individual held in memory. In primates, faces play a particularly important role on the sensory side of this process, which is reflected in an extensive network of face-selective areas along the inferior temporal lobe. However, where and how memory is re-activated during face perception remains unclear. Using functional magnetic resonance imaging (fMRI), we measured whole brain activity in macaques while they were watching pictures of other monkey faces that were either long-term acquaintances, visually familiar, or totally unfamiliar. In comparison to unfamiliar faces, the entire face-processing network showed increased activity in response to familiar faces of long-time personal acquaintances. In contrast, faces that were only visually familiar elicited less activity than totally unfamiliar faces in most face-selective areas. The face-processing network thus distinguished personally familiar faces from visually familiar faces. Personally familiar faces also prompted the activation of two previously unknown face-selective areas in the temporal lobe. One area was located in the perirhinal cortex (PR), which has been associated with declarative memory, and the other area was embedded in the temporal pole (TP), a region previously associated with social knowledge. These two novel face areas showed a non-linear response as blurred faces became gradually visible, rapidly becoming active when the faces of personal acquaintances became recognizable. Thus, mimicking the perception of a face approaching us, this paradigm revealed a neural correlate of the ‘aha!’ recognition moment in face areas TP and PR. As a first step towards advancing our understanding of the neuronal processing of individual recognition, our fMRI experiments identified two novel face areas specifically involved in recognizing familiar faces. However, the hemodynamic response cannot directly assess neurophysiological properties. Using fMRI-guided electrophysiology, we investigated the responses of neurons within the novel face area TP in awake monkeys, and we provided the first systematic evidence of cells selective for familiar faces. A high fraction of neurons in face area TP were selective for familiar monkey faces, and unfamiliar faces that were physically similar failed to elicit the same neural responses. Importantly, neurons in face area AM, which is thought to compute facial identity at the top of the face perception hierarchy, were not modulated by familiarity. Within TP, neurons also responded to monkey bodies, and to monkey vocalizations. Maximum activity was elicited by the joint observation of faces and bodies, and audiovisual interactions were evident in some TP neurons. Together, these results reveal neuronal processes underlying memory re-activation during face perception and generate hypotheses for testing how individual recognition is achieved through different modalities, thus advancing our understanding into how unique representations of familiar individuals are developed at the neural level

Social communication in domestic horses: the production and perception of facial expressions

Living in complex societies is thought to promote the development of sophisticated social, cognitive, and communicative skills. Investigating the extent of these skills across taxa is critical to understanding the evolution of the advanced abilities found in some species, including humans. Facial expressions are rich sources of social information for humans and some primates; however whether this is true for other animals is largely unknown. Horses are an ideal study species for these questions: they form valuable social relationships and display some advanced socio-cognitive skills, but are phylogenetically distant from primates and so might be expected to communicate quite differently. Here I present a method for quantifying and coding horse facial movements (EquiFACS), which reveals that horses have an extensive capacity for producing facial expressions. I then utilise EquiFACS to demonstrate that horses produce facial actions that mirror the emotional content of auditory stimuli, providing evidence for a perception-action representation of emotional information. Through my experiments on the perception of facial expressions in horses I show that these expressions display meaningful information to conspecifics, which influences their behaviour in functionally relevant ways. I also shed light on the physiological processes involved in the perception of emotional conspecific facial expressions, showing that viewing negatively valenced conspecific emotional expressions raises resting heart rate. This is indicative of emotional contagion, which may be a mechanism through which information is obtained and social interactions are regulated. Collectively, my research demonstrates the ability to produce and use complex facial expressions as a source of social information is not limited to primates, but is present in at least two phylogenetically distant groups. This suggests these skills may either be an evolutionarily conserved trait or have evolved under common selective pressures. As well as their scientific significance, these findings have implications for horse management and welfare

Face evaluation: an embodied cognitive approach

American Psychological Association (PsycINFO Classification Categories and Codes) 2320 Sensory Perception 2340 Cognitive Processes 3000 Social Psychology 3040 Social Perception and Cognitionthesis intends to demonstrate that face evaluation can be embodied. Studies in the area of face evaluation suggest that face perception is linked to action. Given that actions depend on our bodies, face evaluation supposedly influence how our bodies will act. Furthermore, I intend to show that the action of our bodies can also influence face perception. In the first three empirical chapters, I tested the notion that judgments of social dominance result from an overgeneralization of properties that signal to the perceiver the potential for someone to act (i.e. physical strength). This may potentially influence our bodily actions. The results indicate that judgments of physical strength predict social dominance. In the last three empirical chapters, I tried to show that bodily actions can influence face evaluation. I showed that an expansive posture can reduce the perception of differences between facial levels of social dominance when compared to a constrictive posture. Participants in the expansive body posture also recreated a mental image of their self-face evidencing greater dominance than participants in a constrictive posture. Finally, I also demonstrated that the interaction between bodies through a multisensory stimulation can influence judgments and the recognition of trustworthiness in faces. Thus, the present thesis shows that face evaluation is embodiedEsta tese tem por objectivo demonstrar que a avaliação de faces pode ser corporalizada. Os estudos na área da avaliação de faces sugerem que a percepção facial está associada à acção. Sendo que as acções dependem dos nossos corpos, a avaliação de faces supostamente influenciará a forma como os nossos corpos irão actuar. Para além disso, pretende-se evidenciar que a acção corporal também pode influenciar a avaliação de faces. Nos três primeiros capítulos empíricos testou-se a noção de que os julgamentos de dominância social resultam de uma generalização de propriedades que sinalizam ao percipiente o potencial de alguém para agir (i.e. força física) o que poderá influenciar as acções corporais. Os resultados indicam que os julgamentos de força física predizem a dominância social. Nos últimos três capítulos empíricos, procurei demonstrar que as acções corporais podem influenciar a avaliação de faces. Demonstrei que uma postura expansiva em comparação com uma postura constritiva pode reduzir a percepção das diferenças entre os níveis faciais de dominância social. Os participantes numa postura corporal expansiva recriaram uma imagem mental do seu eu facial que evidencia uma maior dominância, comparativamente aos participantes numa postura constritiva. Finalmente, evidenciei que a interacção entre os corpos através de uma estimulação multissensorial pode influenciar os julgamentos e o reconhecimento do traço confiável nas faces. Assim, a presente investigação evidencia que a avaliação de faces é corporalizada

Face recognition by means of advanced contributions in machine learning

Face recognition (FR) has been extensively studied, due to both scientific fundamental challenges and current and potential applications where human identification is needed. FR systems have the benefits of their non intrusiveness, low cost of equipments and no useragreement requirements when doing acquisition, among the most important ones. Nevertheless, despite the progress made in last years and the different solutions proposed, FR performance is not yet satisfactory when more demanding conditions are required (different viewpoints, blocked effects, illumination changes, strong lighting states, etc). Particularly, the effect of such non-controlled lighting conditions on face images leads to one of the strongest distortions in facial appearance. This dissertation addresses the problem of FR when dealing with less constrained illumination situations. In order to approach the problem, a new multi-session and multi-spectral face database has been acquired in visible, Near-infrared (NIR) and Thermal infrared (TIR) spectra, under different lighting conditions. A theoretical analysis using information theory to demonstrate the complementarities between different spectral bands have been firstly carried out. The optimal exploitation of the information provided by the set of multispectral images has been subsequently addressed by using multimodal matching score fusion techniques that efficiently synthesize complementary meaningful information among different spectra. Due to peculiarities in thermal images, a specific face segmentation algorithm has been required and developed. In the final proposed system, the Discrete Cosine Transform as dimensionality reduction tool and a fractional distance for matching were used, so that the cost in processing time and memory was significantly reduced. Prior to this classification task, a selection of the relevant frequency bands is proposed in order to optimize the overall system, based on identifying and maximizing independence relations by means of discriminability criteria. The system has been extensively evaluated on the multispectral face database specifically performed for our purpose. On this regard, a new visualization procedure has been suggested in order to combine different bands for establishing valid comparisons and giving statistical information about the significance of the results. This experimental framework has more easily enabled the improvement of robustness against training and testing illumination mismatch. Additionally, focusing problem in thermal spectrum has been also addressed, firstly, for the more general case of the thermal images (or thermograms), and then for the case of facialthermograms from both theoretical and practical point of view. In order to analyze the quality of such facial thermograms degraded by blurring, an appropriate algorithm has been successfully developed. Experimental results strongly support the proposed multispectral facial image fusion, achieving very high performance in several conditions. These results represent a new advance in providing a robust matching across changes in illumination, further inspiring highly accurate FR approaches in practical scenarios.El reconeixement facial (FR) ha estat àmpliament estudiat, degut tant als reptes fonamentals científics que suposa com a les aplicacions actuals i futures on requereix la identificació de les persones. Els sistemes de reconeixement facial tenen els avantatges de ser no intrusius,presentar un baix cost dels equips d’adquisició i no la no necessitat d’autorització per part de l’individu a l’hora de realitzar l'adquisició, entre les més importants. De totes maneres i malgrat els avenços aconseguits en els darrers anys i les diferents solucions proposades, el rendiment del FR encara no resulta satisfactori quan es requereixen condicions més exigents (diferents punts de vista, efectes de bloqueig, canvis en la il·luminació, condicions de llum extremes, etc.). Concretament, l'efecte d'aquestes variacions no controlades en les condicions d'il·luminació sobre les imatges facials condueix a una de les distorsions més accentuades sobre l'aparença facial. Aquesta tesi aborda el problema del FR en condicions d'il·luminació menys restringides. Per tal d'abordar el problema, hem adquirit una nova base de dades de cara multisessió i multiespectral en l'espectre infraroig visible, infraroig proper (NIR) i tèrmic (TIR), sota diferents condicions d'il·luminació. En primer lloc s'ha dut a terme una anàlisi teòrica utilitzant la teoria de la informació per demostrar la complementarietat entre les diferents bandes espectrals objecte d’estudi. L'òptim aprofitament de la informació proporcionada pel conjunt d'imatges multiespectrals s'ha abordat posteriorment mitjançant l'ús de tècniques de fusió de puntuació multimodals, capaces de sintetitzar de manera eficient el conjunt d’informació significativa complementària entre els diferents espectres. A causa de les característiques particulars de les imatges tèrmiques, s’ha requerit del desenvolupament d’un algorisme específic per la segmentació de les mateixes. En el sistema proposat final, s’ha utilitzat com a eina de reducció de la dimensionalitat de les imatges, la Transformada del Cosinus Discreta i una distància fraccional per realitzar les tasques de classificació de manera que el cost en temps de processament i de memòria es va reduir de forma significa. Prèviament a aquesta tasca de classificació, es proposa una selecció de les bandes de freqüències més rellevants, basat en la identificació i la maximització de les relacions d'independència per mitjà de criteris discriminabilitat, per tal d'optimitzar el conjunt del sistema. El sistema ha estat àmpliament avaluat sobre la base de dades de cara multiespectral, desenvolupada pel nostre propòsit. En aquest sentit s'ha suggerit l’ús d’un nou procediment de visualització per combinar diferents bandes per poder establir comparacions vàlides i donar informació estadística sobre el significat dels resultats. Aquest marc experimental ha permès més fàcilment la millora de la robustesa quan les condicions d’il·luminació eren diferents entre els processos d’entrament i test. De forma complementària, s’ha tractat la problemàtica de l’enfocament de les imatges en l'espectre tèrmic, en primer lloc, pel cas general de les imatges tèrmiques (o termogrames) i posteriorment pel cas concret dels termogrames facials, des dels punt de vista tant teòric com pràctic. En aquest sentit i per tal d'analitzar la qualitat d’aquests termogrames facials degradats per efectes de desenfocament, s'ha desenvolupat un últim algorisme. Els resultats experimentals recolzen fermament que la fusió d'imatges facials multiespectrals proposada assoleix un rendiment molt alt en diverses condicions d’il·luminació. Aquests resultats representen un nou avenç en l’aportació de solucions robustes quan es contemplen canvis en la il·luminació, i esperen poder inspirar a futures implementacions de sistemes de reconeixement facial precisos en escenaris no controlats.Postprint (published version

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task