Real-time Hand Gesture Detection and Classification Using Convolutional Neural Networks

Real-time recognition of dynamic hand gestures from video streams is a challenging task since (i) there is no indication when a gesture starts and ends in the video, (ii) performed gestures should only be recognized once, and (iii) the entire architecture should be designed considering the memory and power budget. In this work, we address these challenges by proposing a hierarchical structure enabling offline-working convolutional neural network (CNN) architectures to operate online efficiently by using sliding window approach. The proposed architecture consists of two models: (1) A detector which is a lightweight CNN architecture to detect gestures and (2) a classifier which is a deep CNN to classify the detected gestures. In order to evaluate the single-time activations of the detected gestures, we propose to use Levenshtein distance as an evaluation metric since it can measure misclassifications, multiple detections, and missing detections at the same time. We evaluate our architecture on two publicly available datasets - EgoGesture and NVIDIA Dynamic Hand Gesture Datasets - which require temporal detection and classification of the performed hand gestures. ResNeXt-101 model, which is used as a classifier, achieves the state-of-the-art offline classification accuracy of 94.04% and 83.82% for depth modality on EgoGesture and NVIDIA benchmarks, respectively. In real-time detection and classification, we obtain considerable early detections while achieving performances close to offline operation. The codes and pretrained models used in this work are publicly available.Comment: Published at IEEE International Conference on Automatic Face and Gesture Recognition (FG 2019) - Best student paper award!

Reliable Multimodal Trajectory Prediction via Error Aligned Uncertainty Optimization

Reliable uncertainty quantification in deep neural networks is very crucial in safety-critical applications such as automated driving for trustworthy and informed decision-making. Assessing the quality of uncertainty estimates is challenging as ground truth for uncertainty estimates is not available. Ideally, in a well-calibrated model, uncertainty estimates should perfectly correlate with model error. We propose a novel error aligned uncertainty optimization method and introduce a trainable loss function to guide the models to yield good quality uncertainty estimates aligning with the model error. Our approach targets continuous structured prediction and regression tasks, and is evaluated on multiple datasets including a large-scale vehicle motion prediction task involving real-world distributional shifts. We demonstrate that our method improves average displacement error by 1.69% and 4.69%, and the uncertainty correlation with model error by 17.22% and 19.13% as quantified by Pearson correlation coefficient on two state-of-the-art baselines.Comment: Accepted to ECCV 2022 workshop - Safe Artificial Intelligence for Automated Drivin

BEA: Revisiting anchor-based object detection DNN using Budding Ensemble Architecture

This paper introduces the Budding Ensemble Architecture (BEA), a novel reduced ensemble architecture for anchor-based object detection models. Object detection models are crucial in vision-based tasks, particularly in autonomous systems. They should provide precise bounding box detections while also calibrating their predicted confidence scores, leading to higher-quality uncertainty estimates. However, current models may make erroneous decisions due to false positives receiving high scores or true positives being discarded due to low scores. BEA aims to address these issues. The proposed loss functions in BEA improve the confidence score calibration and lower the uncertainty error, which results in a better distinction of true and false positives and, eventually, higher accuracy of the object detection models. Both Base-YOLOv3 and SSD models were enhanced using the BEA method and its proposed loss functions. The BEA on Base-YOLOv3 trained on the KITTI dataset results in a 6% and 3.7% increase in mAP and AP50, respectively. Utilizing a well-balanced uncertainty estimation threshold to discard samples in real-time even leads to a 9.6% higher AP50 than its base model. This is attributed to a 40% increase in the area under the AP50-based retention curve used to measure the quality of calibration of confidence scores. Furthermore, BEA-YOLOV3 trained on KITTI provides superior out-of-distribution detection on Citypersons, BDD100K, and COCO datasets compared to the ensembles and vanilla models of YOLOv3 and Gaussian-YOLOv3.Comment: 14 pages, 5 pages supplementary material. Accepted at BMVC-202

Genome-wide association study identifies variants in the MHC class I, IL10, and IL23R-IL12RB2 regions associated with Behcet's disease

Behcet's disease is a genetically complex disease of unknown etiology characterized by recurrent inflammatory attacks affecting the orogenital mucosa, eyes and skin. We performed a genome-wide association study with 311,459 SNPs in 1,215 individuals with Behcet's disease (cases) and 1,278 healthy controls from Turkey. We confirmed the known association of Behcet's disease with HLA-B*51 and identified a second, independent association within the MHC Class I region. We also identified an association at IL10 (rs1518111, P = 1.88 x 10(-8)). Using a meta-analysis with an additional five cohorts from Turkey, the Middle East, Europe and Asia, comprising a total of 2,430 cases and 2,660 controls, we identified associations at IL10 (rs1518111, P = 3.54 x 10(-18), odds ratio = 1.45, 95% CI 1.34-1.58) and the IL23R-IL12RB2 locus (rs924080, P = 6.69 x 10(-9), OR = 1.28, 95% CI 1.18-1.39). The disease-associated IL10 variant (the rs1518111 A allele) was associated with diminished mRNA expression and low protein production

Spoofing and disguise variations in face recognition

La Reconnaissance automatique des personnes est devenue un sujet de plus en plus important avec l'augmentation constante des besoins en sécurité. De nombreux systèmes biométriques existent. Ils utilisent différentes caractéristiques humaines. Parmi tous les traits biométriques, la reconnaissance faciale inclut des aspects positifs en termes d'accessibilité et de fiabilité. Dans cette thèse, deux défis en reconnaissance faciales sont étudiés. Le premier est le leurrage. Le leurrage en reconnaissance faciale est présenté. Des contre-mesures permettant d'améliorer les systèmes actuels sont proposés. A cet effet, les attaques basées sur des photographies 2D ou des masques 3D sont analysées. Le second défi exploré dans cette thèse est lié aux variations dues à des altérations du visage (i.e. chirurgie plastique), maquillage et accessoires pour le visage (e.g. occultations par la présence de lunettes). L'impact de ces variations en reconnaissance de visage est étudiée séparément. Ensuite, des techniques robustes contre les variations de camouflage sont proposées.Human recognition has become an important topic as the need and investments for security applications grow continuously. Numerous biometric systems exist which utilize various human characteristics. Among all biometrics traits, face recognition is advantageous in terms of accessibility and reliability. In the thesis, two challenges in face recognition are analyzed. The first one is face spoofing. Spoofing in face recognition is explained together with the countermeasure techniques that are proposed for the protection of face recognition systems against spoofing attacks. For this purpose, both 2D photograph and 3D mask attacks are analyzed. The second challenge explored in the thesis is disguise variations, which are due to facial alterations, facial makeup and facial accessories (occlusions). The impact of these disguise variations on face recognition is explored, separately. Then, techniques which are robust against disguise variations are proposed