PCA-RECT: An Energy-efficient Object Detection Approach for Event Cameras

We present the first purely event-based, energy-efficient approach for object detection and categorization using an event camera. Compared to traditional frame-based cameras, choosing event cameras results in high temporal resolution (order of microseconds), low power consumption (few hundred mW) and wide dynamic range (120 dB) as attractive properties. However, event-based object recognition systems are far behind their frame-based counterparts in terms of accuracy. To this end, this paper presents an event-based feature extraction method devised by accumulating local activity across the image frame and then applying principal component analysis (PCA) to the normalized neighborhood region. Subsequently, we propose a backtracking-free k-d tree mechanism for efficient feature matching by taking advantage of the low-dimensionality of the feature representation. Additionally, the proposed k-d tree mechanism allows for feature selection to obtain a lower-dimensional dictionary representation when hardware resources are limited to implement dimensionality reduction. Consequently, the proposed system can be realized on a field-programmable gate array (FPGA) device leading to high performance over resource ratio. The proposed system is tested on real-world event-based datasets for object categorization, showing superior classification performance and relevance to state-of-the-art algorithms. Additionally, we verified the object detection method and real-time FPGA performance in lab settings under non-controlled illumination conditions with limited training data and ground truth annotations.Comment: Accepted in ACCV 2018 Workshops, to appea

Face authentication on mobile devices: optimization techniques and applications.

Pun Kwok Ho.Thesis (M.Phil.)--Chinese University of Hong Kong, 2005.Includes bibliographical references (leaves 106-111).Abstracts in English and Chinese.Chapter 1. --- Introduction --- p.1Chapter 1.1 --- Background --- p.1Chapter 1.1.1 --- Introduction to Biometrics --- p.1Chapter 1.1.2 --- Face Recognition in General --- p.2Chapter 1.1.3 --- Typical Face Recognition Systems --- p.4Chapter 1.1.4 --- Face Database and Evaluation Protocol --- p.5Chapter 1.1.5 --- Evaluation Metrics --- p.7Chapter 1.1.6 --- Characteristics of Mobile Devices --- p.10Chapter 1.2 --- Motivation and Objectives --- p.12Chapter 1.3 --- Major Contributions --- p.13Chapter 1.3.1 --- Optimization Framework --- p.13Chapter 1.3.2 --- Real Time Principal Component Analysis --- p.14Chapter 1.3.3 --- Real Time Elastic Bunch Graph Matching --- p.14Chapter 1.4 --- Thesis Organization --- p.15Chapter 2. --- Related Work --- p.16Chapter 2.1 --- Face Recognition for Desktop Computers --- p.16Chapter 2.1.1 --- Global Feature Based Systems --- p.16Chapter 2.1.2 --- Local Feature Based Systems --- p.18Chapter 2.1.3 --- Commercial Systems --- p.20Chapter 2.2 --- Biometrics on Mobile Devices --- p.22Chapter 3. --- Optimization Framework --- p.24Chapter 3.1 --- Introduction --- p.24Chapter 3.2 --- Levels of Optimization --- p.25Chapter 3.2.1 --- Algorithm Level --- p.25Chapter 3.2.2 --- Code Level --- p.26Chapter 3.2.3 --- Instruction Level --- p.27Chapter 3.2.4 --- Architecture Level --- p.28Chapter 3.3 --- General Optimization Workflow --- p.29Chapter 3.4 --- Summary --- p.31Chapter 4. --- Real Time Principal Component Analysis --- p.32Chapter 4.1 --- Introduction --- p.32Chapter 4.2 --- System Overview --- p.33Chapter 4.2.1 --- Image Preprocessing --- p.33Chapter 4.2.2 --- PCA Subspace Training --- p.34Chapter 4.2.3 --- PCA Subspace Projection --- p.36Chapter 4.2.4 --- Template Matching --- p.36Chapter 4.3 --- Optimization using Fixed-point Arithmetic --- p.37Chapter 4.3.1 --- Profiling Analysis --- p.37Chapter 4.3.2 --- Fixed-point Representation --- p.38Chapter 4.3.3 --- Range Estimation --- p.39Chapter 4.3.4 --- Code Conversion --- p.42Chapter 4.4 --- Experiments and Discussions --- p.43Chapter 4.4.1 --- Experiment Setup --- p.43Chapter 4.4.2 --- Execution Time --- p.44Chapter 4.4.3 --- Space Requirement --- p.45Chapter 4.4.4 --- Verification Accuracy --- p.45Chapter 5. --- Real Time Elastic Bunch Graph Matching --- p.49Chapter 5.1 --- Introduction --- p.49Chapter 5.2 --- System Overview --- p.50Chapter 5.2.1 --- Image Preprocessing --- p.50Chapter 5.2.2 --- Landmark Localization --- p.51Chapter 5.2.3 --- Feature Extraction --- p.52Chapter 5.2.4 --- Template Matching --- p.53Chapter 5.3 --- Optimization Overview --- p.54Chapter 5.3.1 --- Computation Optimization --- p.55Chapter 5.3.2 --- Memory Optimization --- p.56Chapter 5.4 --- Optimization Strategies --- p.58Chapter 5.4.1 --- Fixed-point Arithmetic --- p.60Chapter 5.4.2 --- Gabor Masks and Bunch Graphs Precomputation --- p.66Chapter 5.4.3 --- Improving Array Access Efficiency using ID array --- p.68Chapter 5.4.4 --- Efficient Gabor Filter Selection --- p.75Chapter 5.4.5 --- Fine Tuning System Cache Policy --- p.79Chapter 5.4.6 --- Reducing Redundant Memory Access by Loop Merging --- p.80Chapter 5.4.7 --- Maximizing Cache Reuse by Array Merging --- p.90Chapter 5.4.8 --- Optimization of Trigonometric Functions using Table Lookup. --- p.97Chapter 5.5 --- Summary --- p.99Chapter 6. --- Conclusions --- p.103Chapter 7. --- Bibliography --- p.10

Low-power dynamic object detection and classification with freely moving event cameras

We present the first purely event-based, energy-efficient approach for dynamic object detection and categorization with a freely moving event camera. Compared to traditional cameras, event-based object recognition systems are considerably behind in terms of accuracy and algorithmic maturity. To this end, this paper presents an event-based feature extraction method devised by accumulating local activity across the image frame and then applying principal component analysis (PCA) to the normalized neighborhood region. Subsequently, we propose a backtracking-free k-d tree mechanism for efficient feature matching by taking advantage of the low-dimensionality of the feature representation. Additionally, the proposed k-d tree mechanism allows for feature selection to obtain a lower-dimensional object representation when hardware resources are limited to implement PCA. Consequently, the proposed system can be realized on a field-programmable gate array (FPGA) device leading to high performance over resource ratio. The proposed system is tested on real-world event-based datasets for object categorization, showing superior classification performance compared to state-of-the-art algorithms. Additionally, we verified the real-time FPGA performance of the proposed object detection method, trained with limited data as opposed to deep learning methods, under a closed-loop aerial vehicle flight mode. We also compare the proposed object categorization framework to pre-trained convolutional neural networks using transfer learning and highlight the drawbacks of using frame-based sensors under dynamic camera motion. Finally, we provide critical insights about the feature extraction method and the classification parameters on the system performance, which aids in understanding the framework to suit various low-power (less than a few watts) application scenarios

Facial emotion recognition using min-max similarity classifier

Recognition of human emotions from the imaging templates is useful in a wide variety of human-computer interaction and intelligent systems applications. However, the automatic recognition of facial expressions using image template matching techniques suffer from the natural variability with facial features and recording conditions. In spite of the progress achieved in facial emotion recognition in recent years, the effective and computationally simple feature selection and classification technique for emotion recognition is still an open problem. In this paper, we propose an efficient and straightforward facial emotion recognition algorithm to reduce the problem of inter-class pixel mismatch during classification. The proposed method includes the application of pixel normalization to remove intensity offsets followed-up with a Min-Max metric in a nearest neighbor classifier that is capable of suppressing feature outliers. The results indicate an improvement of recognition performance from 92.85% to 98.57% for the proposed Min-Max classification method when tested on JAFFE database. The proposed emotion recognition technique outperforms the existing template matching methods

Unsupervised feature learning with discriminative encoder

In recent years, deep discriminative models have achieved extraordinary performance on supervised learning tasks, significantly outperforming their generative counterparts. However, their success relies on the presence of a large amount of labeled data. How can one use the same discriminative models for learning useful features in the absence of labels? We address this question in this paper, by jointly modeling the distribution of data and latent features in a manner that explicitly assigns zero probability to unobserved data. Rather than maximizing the marginal probability of observed data, we maximize the joint probability of the data and the latent features using a two step EM-like procedure. To prevent the model from overfitting to our initial selection of latent features, we use adversarial regularization. Depending on the task, we allow the latent features to be one-hot or real-valued vectors and define a suitable prior on the features. For instance, one-hot features correspond to class labels and are directly used for the unsupervised and semi-supervised classification task, whereas real-valued feature vectors are fed as input to simple classifiers for auxiliary supervised discrimination tasks. The proposed model, which we dub discriminative encoder (or DisCoder), is flexible in the type of latent features that it can capture. The proposed model achieves state-of-the-art performance on several challenging tasks.Comment: 10 pages, 4 figures, International Conference on Data Mining, 201