Novel Deep Learning Techniques For Computer Vision and Structure Health Monitoring

This thesis proposes novel techniques in building a generic framework for both the regression and classification tasks in vastly different applications domains such as computer vision and civil engineering. Many frameworks have been proposed and combined into a complex deep network design to provide a complete solution to a wide variety of problems. The experiment results demonstrate significant improvements of all the proposed techniques towards accuracy and efficiency

치상회 손상에 따른 CA3 장소세포의 장면의존적 발화율 변조의 저하

학위논문 (박사) -- 서울대학교 대학원 : 자연과학대학 뇌인지과학과, 2020. 8. 이인아.The ability to differentiate similar experiences into discrete events in memory is a fundamental component of the episodic memory. Computational models and experimental evidence have suggested that projections from the dentate gyrus (DG) to CA3 play important roles in representing orthogonal information (i.e., pattern separation) in the hippocampus. However, the effects of eliminating the DG on neural firing patterns in the CA3 have rarely been tested in a goal-directed memory task that requires both the DG and CA3. In this thesis, the simultaneous application of lesion and in-vivo electrophysiology were used to examine the role of the DG inputs to the CA3 as the animal processes scene memory. Selective lesions in the DG were made using colchicine in male Long-Evans rats, and CA3 single units were recorded as the rats performed visual scene memory tasks. The original scenes used in training were modified during testing by blurring to varying degrees, by using visual masks, or by overlaying competing scenes to examine how changes in scenes differentially recruit the DG-CA3 circuits. Compared with controls, the performance of rats with DG lesions was particularly impaired when blurred scenes were used in the task. The firing-rate modulation associated with visual scenes in these rats was significantly reduced in the single units recorded from the CA3 when blurred scenes were presented, largely because DG-deprived CA3 cells did not show stepwise, categorical rate changes across varying degrees of scene ambiguity compared with controls. These findings suggest that the DG plays key roles not only during the acquisition of scene memories but also when modified visual scenes are processed in conjunction with the CA3 by making the CA3 network respond orthogonally to ambiguous scenes.특정 환경에 적합한 행동을 하기 위해 우리는 과거에 그 환경에서 어떠한 경험을 하였는지 반추한다. 해마는 이와 같이 과거에 경험한 사건 등에 대한 일화 기억을 처리할 때 중요한 뇌 영역이라고 알려져 있다. 항상 변화하는 환경 속에서 매번 과거와 동일한 자극을 경험하긴 어렵다. 따라서, 일화 기억의 처리에는 서로 유사하지만 다른 상황을 구분하는 과정이 필요하다. 해마의 하위 영역인 치상회는 서로 유사한 자극을 분리하여 저장하기 위해 필요하다고 알려져 있으나, 단위신경세포 수준의 정보처리 기전은 아직 알려진 바 없다. 이 논문에서는 최근 일화 기억의 저장과 인출 과정이 시각적으로 들어오는 장면에 대한 처리와 밀접한 연관을 가지고 있다는 주장에 기반하여, 유사한 장면 자극들이 해마에서 처리되는 기전에 대한 연구를 진행하였다. 이 연구에서는 콜치신을 사용한 쥐의 치상회의 손상이 시각 장면 기억 과제의 수행을 저해하는지 그 행동적 영향을 검증하였다. 또한, 치상회로부터 정보를 받는 CA3영역의 단위신경세포의 활동 전위가 어떠한 방식으로 장면 자극을 표상하고, 기존에 학습했던 장면 자극이 변형되었을 때는 어떻게 표상하는지, 그리고 그 표상 방식들이 치상회의 손상 여부에 따라 어떻게 영향을 받는지 전기생리학적 검증을 시도하였다. 통제 집단에 비해 치상회 손상 집단은 기존에 학습하였던 장면 자극이 흐릿하게 제시될 때에만 낮은 과제 수행률을 보였다. 동시에 치상회가 손상된 쥐들의 CA3 단위신경세포의 시각 장면 관련 발화율 변조가 흐릿한 장면 자극이 제시되었을 때 현저하게 감소하였다. 이는 치상회가 손상된 쥐의 CA3 단위신경세포가 통제 집단에 비해 흐릿한 장면 자극의 모호성의 정도에 따른 범주적 변화를 보여주지 않았기 때문이다. 본 연구는 치상회가 CA3 영역이 애매한 장면에 직교적으로 반응하게 함으로써 장면 기억을 저장하는 과정뿐만 아니라, CA3와 함께 수정된 시각 장면이 처리될 때에도 핵심적인 역할을 한다는 것을 규명하였다.Chapter 1. Recognition of ambiguous visual scenes following the lesion of the dentate gyrus 12 Introduction 13 Materials and methods 14 Results 26 Discussion 33 Chapter 2. Impaired pattern separation in scene-dependent rate modulation in CA3 single units following the lesion of the dentate gyrus 36 Introduction 37 Materials and methods 38 Results 49 Discussion 75 General Discussion 77 Bibliography 85 Acknowledgement (감사의 말) 97 국문초록 98Docto

Markerless Motion Capture via Convolutional Neural Network

A human motion capture system can be defined as a process that digitally records the movements of a person and then translates them into computer-animated images. To achieve this goal, motion capture systems usually exploit different types of algorithms, which include techniques such as pose estimation or background subtraction: this latter aims at segmenting moving objects from the background under multiple challenging scenarios. Recently, encoder-decoder-type deep neural networks designed to accomplish this task have reached impressive results, outperforming classical approaches. The aim of this thesis is to evaluate and discuss the predictions provided by the multi-scale convolutional neural network FgSegNet_v2, a deep learning-based method which represents the current state-of-the-art for implementing scene-specific background subtraction. In this work, FgSegNet_v2 is trained and tested on BBSoF S.r.l. dataset, extending its scene- specific use to a more general application in several environments

Learning Smooth Pattern Transformation Manifolds

Manifold models provide low-dimensional representations that are useful for processing and analyzing data in a transformation-invariant way. In this paper, we study the problem of learning smooth pattern transformation manifolds from image sets that are observations of geometrically transformed signals. In order to construct a manifold, we build a representative pattern whose transformations accurately fit various input images. The pattern is formed by selecting a good common sparse approximation of the images with parametric and smooth atoms. We examine two aspects of the manifold building problem, where we first target an accurate transformation-invariant approximation of the input images, and then extend this solution for their classification. For the approximation problem, we propose a greedy method that constructs a representative pattern by selecting analytic atoms from a continuous dictionary manifold. We present a DC (Difference-of-Convex) optimization scheme which is applicable for a wide range of transformation and dictionary models, and demonstrate its application to transformation manifolds generated by the rotation, translation and anisotropic scaling of a reference pattern. Then, we generalize this approach to a setting with multiple transformation manifolds, where each manifold represents a different class of signals. We present an iterative multiple manifold building algorithm such that the classification accuracy is promoted in the joint selection of atoms. Experimental results suggest that the proposed methods yield high accuracy in the approximation and classification of data in comparison with some reference methods, while achieving invariance to geometric transformations due to the transformation manifold model

RGB to 3D garment reconstruction using UV map representations

Predicting the geometry of a 3D object from just a single image or viewpoint is an intrinsic human feature extremely challenging for machines. For years, in an attempt to solve this problem, different computer vision approaches and techniques have been investigated. One of the domains in which there has been more research has been the 3D reconstruction and modelling of human bodies. However, the greatest advances in this field have concentrated on the recovery of unclothed human bodies, ignoring garments. Garments are highly detailed, dynamic objects made up of particles that interact with each other and with other objects, making the task of reconstruction even more difficult. Therefore, having a lightweight 3D representation capable of modelling fine details is of great importance. This thesis presents a deep learning framework based on Generative Adversarial Networks (GANs) to reconstruct 3D garment models from a single RGB image. It has the peculiarity of using UV maps to represent 3D data, a lightweight representation capable of dealing with high-resolution details and wrinkles. With this model and kind of 3D representation, we achieve state-of-the-art results on CLOTH3D dataset, generating good quality and realistic reconstructions regardless of the garment topology, human pose, occlusions and lightning, and thus demonstrating the suitability of UV maps for 3D domains and tasks

Dynamic and Integrative Properties of the Primary Visual Cortex

The ability to derive meaning from complex, ambiguous sensory input requires the integration of information over both space and time, as well as cognitive mechanisms to dynamically shape that integration. We have studied these processes in the primary visual cortex (V1), where neurons have been proposed to integrate visual inputs along a geometric pattern known as the association field (AF). We first used cortical reorganization as a model to investigate the role that a specific network of V1 connections, the long-range horizontal connections, might play in temporal and spatial integration across the AF. When retinal lesions ablate sensory information from portions of the visual field, V1 undergoes a process of reorganization mediated by compensatory changes in the network of horizontal collaterals. The reorganization accompanies the brain’s amazing ability to perceptually “fill-inâ€, or “seeâ€, the lost visual input. We developed a computational model to simulate cortical reorganization and perceptual fill-in mediated by a plexus of horizontal connections that encode the AF. The model reproduces the major features of the perceptual fill-in reported by human subjects with retinal lesions, and it suggests that V1 neurons, empowered by their horizontal connections, underlie both perceptual fill-in and normal integrative mechanisms that are crucial to our visual perception. These results motivated the second prong of our work, which was to experimentally study the normal integration of information in V1. Since psychophysical and physiological studies suggest that spatial interactions in V1 may be under cognitive control, we investigated the integrative properties of V1 neurons under different cognitive states. We performed extracellular recordings from single V1 neurons in macaques that were trained to perform a delayed-match-to-sample contour detection task. We found that the ability of V1 neurons to summate visual inputs from beyond the classical receptive field (cRF) imbues them with selectivity for complex contour shapes, and that neuronal shape selectivity in V1 changed dynamically according to the shapes monkeys were cued to detect. Over the population, V1 encoded subsets of the AF, predicted by the computational model, that shifted as a function of the monkeys’ expectations. These results support the major conclusions of the theoretical work; even more, they reveal a sophisticated mode of form processing, whereby the selectivity of the whole network in V1 is reshaped by cognitive state

Neuromorphic Engineering Editors' Pick 2021

This collection showcases well-received spontaneous articles from the past couple of years, which have been specially handpicked by our Chief Editors, Profs. André van Schaik and Bernabé Linares-Barranco. The work presented here highlights the broad diversity of research performed across the section and aims to put a spotlight on the main areas of interest. All research presented here displays strong advances in theory, experiment, and methodology with applications to compelling problems. This collection aims to further support Frontiers’ strong community by recognizing highly deserving authors