Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars

Event cameras are bio-inspired vision sensors that naturally capture the dynamics of a scene, filtering out redundant information. This paper presents a deep neural network approach that unlocks the potential of event cameras on a challenging motion-estimation task: prediction of a vehicle's steering angle. To make the best out of this sensor-algorithm combination, we adapt state-of-the-art convolutional architectures to the output of event sensors and extensively evaluate the performance of our approach on a publicly available large scale event-camera dataset (~1000 km). We present qualitative and quantitative explanations of why event cameras allow robust steering prediction even in cases where traditional cameras fail, e.g. challenging illumination conditions and fast motion. Finally, we demonstrate the advantages of leveraging transfer learning from traditional to event-based vision, and show that our approach outperforms state-of-the-art algorithms based on standard cameras.Comment: 9 pages, 8 figures, 6 tables. Video: https://youtu.be/_r_bsjkJTH

Recurrent Attention Models for Depth-Based Person Identification

We present an attention-based model that reasons on human body shape and motion dynamics to identify individuals in the absence of RGB information, hence in the dark. Our approach leverages unique 4D spatio-temporal signatures to address the identification problem across days. Formulated as a reinforcement learning task, our model is based on a combination of convolutional and recurrent neural networks with the goal of identifying small, discriminative regions indicative of human identity. We demonstrate that our model produces state-of-the-art results on several published datasets given only depth images. We further study the robustness of our model towards viewpoint, appearance, and volumetric changes. Finally, we share insights gleaned from interpretable 2D, 3D, and 4D visualizations of our model's spatio-temporal attention.Comment: Computer Vision and Pattern Recognition (CVPR) 201

Parallel computing for brain simulation

[Abstract] Background: The human brain is the most complex system in the known universe, it is therefore one of the greatest mysteries. It provides human beings with extraordinary abilities. However, until now it has not been understood yet how and why most of these abilities are produced. Aims: For decades, researchers have been trying to make computers reproduce these abilities, focusing on both understanding the nervous system and, on processing data in a more efficient way than before. Their aim is to make computers process information similarly to the brain. Important technological developments and vast multidisciplinary projects have allowed creating the first simulation with a number of neurons similar to that of a human brain. Conclusion: This paper presents an up-to-date review about the main research projects that are trying to simulate and/or emulate the human brain. They employ different types of computational models using parallel computing: digital models, analog models and hybrid models. This review includes the current applications of these works, as well as future trends. It is focused on various works that look for advanced progress in Neuroscience and still others which seek new discoveries in Computer Science (neuromorphic hardware, machine learning techniques). Their most outstanding characteristics are summarized and the latest advances and future plans are presented. In addition, this review points out the importance of considering not only neurons: Computational models of the brain should also include glial cells, given the proven importance of astrocytes in information processing.Galicia. Consellería de Cultura, Educación e Ordenación Universitaria; GRC2014/049Galicia. Consellería de Cultura, Educación e Ordenación Universitaria; R2014/039Instituto de Salud Carlos III; PI13/0028

A historical perspective of algorithmic lateral inhibition and accumulative computation in computer vision

Certainly, one of the prominent ideas of Professor José Mira was that it is absolutely mandatory to specify the mechanisms and/or processes underlying each task and inference mentioned in an architecture in order to make operational that architecture. The conjecture of the last fifteen years of joint research has been that any bottom-up organization may be made operational using two biologically inspired methods called ?algorithmic lateral inhibition?, a generalization of lateral inhibition anatomical circuits, and ?accumulative computation?, a working memory related to the temporal evolution of the membrane potential. This paper is dedicated to the computational formulation of both methods. Finally, all of the works of our group related to this methodological approximation are mentioned and summarized, showing that all of them support the validity of this approximation

콘볼루션 신경망의 색상 위계 학습에 대한 탐구

학위논문 (석사) -- 서울대학교 대학원 : 인문대학 협동과정 인지과학전공, 2020. 8. 장병탁.Empirical evidence suggests that color categories emerge in a universal, recurrent, hierarchical pattern across different cultures in the following order; white, black < red < green, yellow < blue < brown < pink, gray, orange, and purple}. This pattern is referred to as the "Color Hierarchy". Over two experiments, the present study examines whether there is evidence for such hierarchical color category learning patterns in Convolutional Neural Networks (CNNs). Experiment A investigates whether color categories are learned randomly, or in a fixed, hierarchical fashion. Results show that colors higher up the Color Hierarchy (e.g. red) are generally learned before colors lower down the hierarchy (e.g. brown, orange, gray). Experiment B examines whether object color affects recall in object detection. Similar to Experiment A, results show that object recall is noticeably impacted by color, with colors higher up the Color Hierarchy generally showing better recall. Additionally, objects whose color can be described by adjectives that emphasise colorfulness (e.g. vivid, brilliant, deep) show better recall than those which de-emphasise colorfulness (e.g. dark, pale, light). The effect of both color hue and adjective on object recall is still observable, even when controlling for contrast through grayscale images. These results highlight similarities between humans and CNNs in color perception, and provide insight into factors that influence object detection. They also show the value of using deep learning techniques as a means of investigating cognitive universalities in an efficient, unbiased, cost-effective way.경험적으로 색상 계열은 보편적으로 다양한 문화권에 걸쳐 순환적이고 위계적인 패턴을 나타내며 그 순서는 다음과 같이 나타난다; 검은색 < 붉은색 <녹색 < 노란색 < 파란색 < 갈색 < 분홍색, 회색, 주황색, 보라색}. 이러한 경향은 "색상 위계(Color Hierarchy)"라 불린다. 소개될 두 가지 실험을 통해 본 연구는 콘볼루션 신경망의 경우에도 색상 위계 순서에 따른 색상계열 학습이 진행되는지 알아본다. 실험 A는 색상 계열이 무작위로 학습되는지 위계적인 순서를 통해 학습되는지 알아본다. 실험의 결과를 통해 색상 위계상으로 더 상위의 색상(예: 붉은색)은 일반적으로 하위의 색상들 (예: 갈색, 주황색, 회색) 보다 앞서 학습이 이뤄짐을 볼 수 있다. 실험 B는 색상 위계에 따른 학습 편차가 객체인식 학습의 재현률(recall)에도 영향을 끼치는지 알아본다. 실험 A에서와같이 색상 위계는 객체인식 재현률에도 큰 영향을 끼친다. 추가적으로 색상을 강조하는 부사(예: 선명한, 눈에 띄는, 짙은)와 함께 묘사된 객체의 경우에는 반대로 색상을 억제하는 부사(예: 어두운, 옅은, 엷은)와 함께 묘사된 객체들보다 재현률이 높게 나타난다. 부사와 색상의 효과는 흑백 이미지들에 대해서도 여전히 관측된다. 이와 같은 결과들은 사람과 콘볼루션 신경망의 색상 지각과정의 유사성을 보여주며 객체 인식에 영향을 주는 요인들에 대한 통찰력을 제공한다. 또한 이 결과들은 딥러닝 방법이 인지과정의 보편성을 살피는 데에 효율적이고, 치우치지 않으며, 경제적인 방법임을 지시한다.Chapter 1 Introduction 1 1.1 Is Color Categorization Random? 2 1.2 Modelling the Color Hierarchy 4 1.3 Convolutional Neural Networks and Color Learning 5 1.4 Hypotheses 6 Chapter 2 Datasets 8 2.1 Basic Color Dataset 8 2.2 Modanet 8 2.2.1 Color Annotating Process 9 Chapter 3 Color Space 12 3.1 Opponent Color Space 12 3.2 Luminance Color Spaces 13 Chapter 4 Experiment A: CNN Color Classification Recall Ex-periment 15 4.1 Model 15 4.2 Method 17 4.3 Results 17 Chapter 5 Experiment B: Faster R-CNN Colored Clothing Recall Experiment 21 5.1 Model 21 5.2 Method 22 5.3 Results 23 Chapter 6 Discussion and Conclusion 28 References 33 국문초록 36 Acknowledgements 37Maste

Revisiting algorithmic lateral inhibition and accumulative computation

Certainly, one of the prominent ideas of Professor Mira was that it is absolutely mandatory to specify the mechanisms and/or processes underlying each task and inference mentioned in an architecture in order to make operational that architecture. The conjecture of the last fifteen years of joint research of Professor Mira and our team at University of Castilla-La Mancha has been that any bottom-up organization may be made operational using two biologically inspired methods called ?algorithmic lateral inhibition?, a generalization of lateral inhibition anatomical circuits, and ?accumulative computation?, a working memory related to the temporal evolution of the membrane potential. This paper is dedicated to the computational formulations of both methods, which have led to quite efficient solutions of problems related to motion-based computer vision