10 research outputs found
Affect Analysis and Membership Recognition in Group Settings
PhD ThesisEmotions play an important role in our day-to-day life in various ways, including, but not
limited to, how we humans communicate and behave. Machines can interact with humans
more naturally and intelligently if they are able to recognise and understand humansā emotions
and express their own emotions. To achieve this goal, in the past two decades, researchers
have been paying a lot of attention to the analysis of affective states, which has been studied
extensively across various fields, such as neuroscience, psychology, cognitive science, and
computer science. Most of the existing works focus on affect analysis in individual settings,
where there is one person in an image or in a video. However, in the real world, people
are very often with others, or interact in group settings. In this thesis, we will focus on
affect analysis in group settings. Affect analysis in group settings is different from that in
individual settings and provides more challenges due to dynamic interactions between the
group members, various occlusions among people in the scene, and the complex context,
e.g., who people are with, where people are staying and the mutual influences among people
in the group. Because of these challenges, there are still a number of open issues that need
further investigation in order to advance the state of the art, and explore the methodologies
for affect analysis in group settings. These open topics include but are not limited to (1) is
it possible to transfer the methods used for the affect recognition of a person in individual
settings to the affect recognition of each individual in group settings? (2) is it possible to
recognise the affect of one individual using the expressed behaviours of another member in the same group (i.e., cross-subject affect recognition)? (3) can non-verbal behaviours be used
for the recognition of contextual information in group settings?
In this thesis, we investigate the affect analysis in group settings and propose methods to
explore the aforementioned research questions step by step. Firstly, we propose a method for
individual affect recognition in both individual and group videos, which is also used for social
context prediction, i.e., whether a person is alone or within a group. Secondly, we introduce
a novel framework for cross-subject affect analysis in group videos. Specifically, we analyse
the correlation of the affect among group members and investigate the automatic recognition
of the affect of one subject using the behaviours expressed by another subject in the same
group or in a different group. Furthermore, we propose methods for contextual information
prediction in group settings, i.e., group membership recognition - to recognise which group
of the person belongs. Comprehensive experiments are conducted using two datasets that
one contains individual videos and one contains group videos. The experimental results show
that (1) the methods used for affect recognition of a person in individual settings can be
transferred to group settings; (2) the affect of one subject in a group can be better predicted
using the expressive behaviours of another subject within the same group than using that of
a subject from a different group; and (3) contextual information (i.e., whether a person is
staying alone or within a group, and group membership) can be predicted successfully using
non-verbal behaviours
Face Alignment Assisted by Head Pose Estimation
In this paper we propose a supervised initialization scheme for cascaded face
alignment based on explicit head pose estimation. We first investigate the
failure cases of most state of the art face alignment approaches and observe
that these failures often share one common global property, i.e. the head pose
variation is usually large. Inspired by this, we propose a deep convolutional
network model for reliable and accurate head pose estimation. Instead of using
a mean face shape, or randomly selected shapes for cascaded face alignment
initialisation, we propose two schemes for generating initialisation: the first
one relies on projecting a mean 3D face shape (represented by 3D facial
landmarks) onto 2D image under the estimated head pose; the second one searches
nearest neighbour shapes from the training set according to head pose distance.
By doing so, the initialisation gets closer to the actual shape, which enhances
the possibility of convergence and in turn improves the face alignment
performance. We demonstrate the proposed method on the benchmark 300W dataset
and show very competitive performance in both head pose estimation and face
alignment.Comment: Accepted by BMVC201
Novel Tactile-SIFT Descriptor for Object Shape Recognition
Using a tactile array sensor to recognize an object often requires multiple touches at different positions. This process is prone to move or rotate the object, which inevitably increases difficulty in object recognition. To cope with the unknown object movement, this paper proposes a new tactile-SIFT descriptor to extract features in view of gradients in the tactile image to represent objects, to allow the features being invariant to object translation and rotation. The tactile-SIFT segments a tactile image into overlapping subpatches, each of which is represented using a dn-dimensional gradient vector, similar to the classic SIFT descriptor. Tactile-SIFT descriptors obtained from multiple touches form a dictionary of k words, and the bag-of-words method is then used to identify objects. The proposed method has been validated by classifying 18 real objects with data from an off-the-shelf tactile sensor. The parameters of the tactile-SIFT descriptor, including the dimension size dn and the number of subpatches sp, are studied. It is found that the optimal performance is obtained using an 8-D descriptor with three subpatches, taking both the classification accuracy and time efficiency into consideration. By employing tactile-SIFT, a recognition rate of 91.33% has been achieved with a dictionary size of 50 clusters using only 15 touches
iCLAP: Shape Recognition by Combining Proprioception and Touch Sensing
The work presented in this paper was partially supported by the Engineering and Physical Sciences Council (EPSRC) Grant (Ref: EP/N020421/1) and the Kingās-China Scholarship Council Ph.D. scholarship
Zwitterionic polymers-armored amyloid-like protein surface combats thrombosis and biofouling
Proteins, cells and bacteria adhering to the surface of medical devices can lead to thrombosis and infection, resulting in significant clinical mortality. Here, we report a zwitterionic polymers-armored amyloid-like protein surface engineering strategy we called as āarmored-tankā strategy for dual functionalization of medical devices. The āarmored-tankā strategy is realized by decoration of partially conformational transformed LZM (PCTL) assembly through oxidant-mediated process, followed by armoring with super-hydrophilic poly-2-methacryloyloxyethyl phosphorylcholine (pMPC). The outer armor of the āarmored-tankā shows potent and durable zone defense against fibrinogen, platelet and bacteria adhesion, leading to long-term antithrombogenic properties over 14 days in vivo without anticoagulation. Additionally, the āfiredā PCTL from āarmored-tankā actively and effectively kills both Gram-positive and Gram-negative bacterial over 30 days as a supplement to the lacking bactericidal functions of passive outer armor. Overall, this āarmored-tankā surface engineering strategy serves as a promising solution for preventing biofouling and thrombotic occlusion of medical devices
Association between the nasopharyngeal microbiome and metabolome in patients with COVID-19
SARS-CoV-2, the causative agent for COVID-19, infect human mainly via respiratory tract, which is heavily inhabited by local microbiota. However, the interaction between SARS-CoV-2 and nasopharyngeal microbiota, and the association with metabolome has not been well characterized. Here, metabolomic analysis of blood, urine, and nasopharyngeal swabs from a group of COVID-19 and non-COVID-19 patients, and metagenomic analysis of pharyngeal samples were used to identify the key features of COVID-19. Results showed lactic acid, l-proline, and chlorogenic acid methyl ester (CME) were significantly reduced in the sera of COVID-19 patients compared with non-COVID-19 ones. Nasopharyngeal commensal bacteria including Gemella morbillorum, Gemella haemolysans and Leptotrichia hofstadii were notably depleted in the pharynges of COVID-19 patients, while Prevotella histicola, Streptococcus sanguinis, and Veillonella dispar were relatively increased. The abundance of G. haemolysans and L. hofstadii were significantly positively associated with serum CME, which might be an anti-SARS-CoV-2 bacterial metabolite. This study provides important information to explore the linkage between nasopharyngeal microbiota and disease susceptibility. The findings were based on a very limited number of patients enrolled in this study; a larger size of cohort will be appreciated for further investigation