Multimodal Emotion Recognition via Convolutional Neural Networks: Comparison of different strategies on two multimodal datasets

The aim of this paper is to investigate emotion recognition using a multimodal approach that exploits convolutional neural networks (CNNs) with multiple input. Multimodal approaches allow different modalities to cooperate in order to achieve generally better performances because different features are extracted from different pieces of information. In this work, the facial frames, the optical flow computed from consecutive facial frames, and the Mel Spectrograms (from the word melody) are extracted from videos and combined together in different ways to understand which modality combination works better. Several experiments are run on the models by first considering one modality at a time so that good accuracy results are found on each modality. Afterward, the models are concatenated to create a final model that allows multiple inputs. For the experiments the datasets used are BAUM-1 ((Bahçeşehir University Multimodal Affective Database - 1) and RAVDESS (Ryerson Audio–Visual Database of Emotional Speech and Song), which both collect two distinguished sets of videos based on the different intensity of the expression, that is acted/strong or spontaneous/normal, providing the representations of the following emotional states that will be taken into consideration: angry, disgust, fearful, happy and sad. The performances of the proposed models are shown through accuracy results and some confusion matrices, demonstrating better accuracy than the compared proposals in the literature. The best accuracy achieved on BAUM-1 dataset is about 95%, while on RAVDESS it is about 95.5%

Plasmonic nanoparticle monomers and dimers: From nano-antennas to chiral metamaterials

We review the basic physics behind light interaction with plasmonic nanoparticles. The theoretical foundations of light scattering on one metallic particle (a plasmonic monomer) and two interacting particles (a plasmonic dimer) are systematically investigated. Expressions for effective particle susceptibility (polarizability) are derived, and applications of these results to plasmonic nanoantennas are outlined. In the long-wavelength limit, the effective macroscopic parameters of an array of plasmonic dimers are calculated. These parameters are attributable to an effective medium corresponding to a dilute arrangement of nanoparticles, i.e., a metamaterial where plasmonic monomers or dimers have the function of "meta-atoms". It is shown that planar dimers consisting of rod-like particles generally possess elliptical dichroism and function as atoms for planar chiral metamaterials. The fabricational simplicity of the proposed rod-dimer geometry can be used in the design of more cost-effective chiral metamaterials in the optical domain.Comment: submitted to Appl. Phys.

Machine Learning for Educational Metaverse: How Far Are We?

The concept of metaverse is becoming pervasive and promises to revolutionize the way people will interact with each other in a sustainable manner. The educational context seems to represent an ideal use case, as the metaverse may provide a digital environment empowered by analytical instruments able to monitor the social and psychological needs of students, other than lowering the entry barriers of students with disabilities. Machine learning will represent a key component of such a new consumer technology, yet little is known about its adoption within an educational metaverse. This paper overviews the current state of the art and provides a discussion about its suitability, in an effort of highlighting future research avenues and challenges