FusionSense: Emotion Classification using Feature Fusion of Multimodal Data and Deep learning in a Brain-inspired Spiking Neural Network

Using multimodal signals to solve the problem of emotion recognition is one of the emerging trends in affective computing. Several studies have utilized state of the art deep learning methods and combined physiological signals, such as the electrocardiogram (EEG), electroencephalogram (ECG), skin temperature, along with facial expressions, voice, posture to name a few, in order to classify emotions. Spiking neural networks (SNNs) represent the third generation of neural networks and employ biologically plausible models of neurons. SNNs have been shown to handle Spatio-temporal data, which is essentially the nature of the data encountered in emotion recognition problem, in an efficient manner. In this work, for the first time, we propose the application of SNNs in order to solve the emotion recognition problem with the multimodal dataset. Specifically, we use the NeuCube framework, which employs an evolving SNN architecture to classify emotional valence and evaluate the performance of our approach on the MAHNOB-HCI dataset. The multimodal data used in our work consists of facial expressions along with physiological signals such as ECG, skin temperature, skin conductance, respiration signal, mouth length, and pupil size. We perform classification under the Leave-One-Subject-Out (LOSO) cross-validation mode. Our results show that the proposed approach achieves an accuracy of 73.15% for classifying binary valence when applying feature-level fusion, which is comparable to other deep learning methods. We achieve this accuracy even without using EEG, which other deep learning methods have relied on to achieve this level of accuracy. In conclusion, we have demonstrated that the SNN can be successfully used for solving the emotion recognition problem with multimodal data and also provide directions for future research utilizing SNN for Affective computing. In addition to the good accuracy, the SNN recognition system is requires incrementally trainable on new data in an adaptive way. It only one pass training, which makes it suitable for practical and on-line applications. These features are not manifested in other methods for this problem.Peer reviewe

Reports on computer graphics testbed to simulate and test vision systems for space applications

Three reports are presented on computer graphics testbed to simulate and test vision systems for space applications

Bit plane slicing technique to classify date varieties

Varietal purity is an important parameter in the quality standards of dates. In general, variety identification is done by visual inspection method in grading and handling facilities. Online variety assessment using computer vision methods with minimum features and fast image processing and classification algorithms would be highly beneficial for the date industry. Three date varieties (Khalas, Fard and Madina) were classified using a single type of feature, Euler number, used on the eight bit planes available from gray scale images. An overall classification accuracy of 91.5% was achieved using a two layer neural network classifier with hyperbolic tangent sigmoid transfer function. Additionally, image segmentation was performed using the two most significant bit planes. Therefore, a complete feature extraction module based on logic values and morphological image processing as proposed here can be easily implemented in hardware

A Review on Facial Expression Recognition Techniques

Facial expression is in the topic of active research over the past few decades. Recognition and extracting various emotions and validating those emotions from the facial expression become very important in human computer interaction. Interpreting such human expression remains and much of the research is required about the way they relate to human affect. Apart from H-I interfaces other applications include awareness system, medical diagnosis, surveillance, law enforcement, automated tutoring system and many more. In the recent year different technique have been put forward for developing automated facial expression recognition system. This paper present quick survey on some of the facial expression recognition techniques. A comparative study is carried out using various feature extraction techniques. We define taxonomy of the field and cover all the steps from face detection to facial expression classification

An efficient approach of face detection and recognition from digital images for modern security and office hour attendance system

This thesis report is submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Science and Engineering, 2015.The purpose of this project is to make an efficient security system for university safety measurement which can also be used to calculate the office hours of Student Tutors by face detection and recognition. By using surveillance cameras, attached at all the entrance of university main buildings, the system can detect human faces and then it can recognize people. First, the system captures the image of a person who enters into the building and then detects the face from the image. Then the recognition system matches that image with the given database of images with valid information. After matching that image if the system recognize that face it gives a green signal to allow that person. Otherwise, if the system cannot recognize that face it gives an alert signal to block that person as an intruder. Also, this system calculates the office hours of the Student Tutors. By using face recognition the system takes the starting time and ending time of the Student Tutors individually and then gives the result as output by calculating the time duration