Real-Time Facial Emotion Recognition Using Fast R-CNN

In computer vision and image processing, object detection algorithms are used to detect semantic objects of certain classes of images and videos. Object detector algorithms use deep learning networks to classify detected regions. Unprecedented advancements in Convolutional Neural Networks (CNN) have led to new possibilities and implementations for object detectors. An object detector which uses a deep learning algorithm detect objects through proposed regions, and then classifies the region using a CNN. Object detectors are computationally efficient unlike a typical CNN which is computationally complex and expensive. Object detectors are widely used for face detection, recognition, and object tracking. In this thesis, deep learning based object detection algorithms are implemented to classify facially expressed emotions in real-time captured through a webcam. A typical CNN would classify images without specifying regions within an image, which could be considered as a limitation towards better understanding the network performance which depend on different training options. It would also be more difficult to verify whether a network have converged and is able to generalize, which is the ability to classify unseen data, data which was not part of the training set. Fast Region-based Convolutional Neural Network, an object detection algorithm; used to detect facially expressed emotion in real-time by classifying proposed regions. The Fast R-CNN is trained using a high-quality video database, consisting of 24 actors, facially expressing eight different emotions, obtained from images which were processed from 60 videos per actor. An object detector’s performance is measured using various metrics. Regardless of how an object detector performed with respect to average precision or miss rate, doing well on such metrics would not necessarily mean that the network is correctly classifying regions. This may result from the fact that the network model has been over-trained. In our work we showed that object detector algorithm such as Fast R-CNN performed surprisingly well in classifying facially expressed emotions in real-time, performing better than CNN

Face recognition for vehicle personalization

The objective of this dissertation is to develop a system of practical technologies to implement an illumination robust, consumer grade biometric system based on face recognition to be used in the automotive market. Most current face recognition systems are compromised in accuracy by ambient illumination changes. Especially outdoor applications including vehicle personalization pose the most challenging environment for face recognition. The point of this research is to investigate practical face recognition used for identity management in order to minimize algorithmic complexity while making the system robust to ambient illumination changes. We start this dissertation by proposing an end-to-end face recognition system using near infrared (NIR) spectrum. The advantage of NIR over visible light is that it is invisible to the human eyes while most CCD and CMOS imaging devices show reasonable response to NIR. Therefore, we can build an unobtrusive night-time vision system with active NIR illumination. In day time the active NIR illumination provides more controlled illumination condition. Next, we propose an end-to-end system with active NIR image differencing which takes the difference between successive image frames, one illuminated and one not illuminated, to make the system more robust on illumination changes. Furthermore, we addresses several aspects of the problem in active NIR image differencing which are motion artifact and noise in the difference frame, namely how to efficiently and more accurately align the illuminated frame and ambient frame, and how to combine information in the difference frame and the illuminated frame. Finally, we conclude the dissertation by citing the contributions of the research and discussing the avenues for future work.Ph.D