Network mechanisms underlying stable motor actions

While we can learn to produce stereotyped movements and maintain this ability for years, it is unclear how populations of individual neurons change their firing properties to coordinate these skills. This has been difficult to address because there is a lack of tools that can monitor populations of single neurons in freely behaving animals for the durations required to remark on their tuning. This thesis is divided into two main directions- device engineering and systems neuroscience. The first section describes the development of an electrode array comprised of tiny self-splaying carbon fibers that are small and flexible enough to avoid the immune response that typically limits electrophysiological recordings. I also describe the refinement of a head-mounted miniature microscope system, optimized for multi-month monitoring of cells expressing genetically encoded calcium indicators in freely behaving animals. In the second section, these tools are used to answer basic systems neuroscience questions in an animal with one of the most stable, complex learned behaviors in the animal kingdom: songbirds. This section explores the functional organization and long-term network stability of HVC, the songbird premotor cortical microcircuit that controls song. Our results reveal that neural activity in HVC is correlated with a length scale of 100um. At this mesocopic scale, basal-ganglia projecting excitatory neurons, on average, fire at a specific phase of a local 30Hz network rhythm. These results show that premotor cortical activity is inhomogeneous in time and space, and that a mesoscopic dynamical pattern underlies the generation of the neural sequences controlling song. At this mesoscopic level, neural coding is stable for weeks and months. These ensemble patterns persist after peripheral nerve damage, revealing that sensory-motor correspondence is not required to maintain the stability of the underlying neural ensemble. However, closer examination of individual excitatory neurons reveals that the participation of cells can change over the timescale of days- with particularly large shifts occurring over instances of sleep. Our findings suggest that fine-scale drift of projection neurons, stabilized by mesoscopic level dynamics dominated by inhibition, forms the mechanistic basis of memory maintenance and and motor stability

DEEP LEARNING FOR IMAGE RESTORATION AND ROBOTIC VISION

Traditional model-based approach requires the formulation of mathematical model, and the model often has limited performance. The quality of an image may degrade due to a variety of reasons: It could be the context of scene is affected by weather conditions such as haze, rain, and snow; It\u27s also possible that there is some noise generated during image processing/transmission (e.g., artifacts generated during compression.). The goal of image restoration is to restore the image back to desirable quality both subjectively and objectively. Agricultural robotics is gaining interest these days since most agricultural works are lengthy and repetitive. Computer vision is crucial to robots especially the autonomous ones. However, it is challenging to have a precise mathematical model to describe the aforementioned problems. Compared with traditional approach, learning-based approach has an edge since it does not require any model to describe the problem. Moreover, learning-based approach now has the best-in-class performance on most of the vision problems such as image dehazing, super-resolution, and image recognition. In this dissertation, we address the problem of image restoration and robotic vision with deep learning. These two problems are highly related with each other from a unique network architecture perspective: It is essential to select appropriate networks when dealing with different problems. Specifically, we solve the problems of single image dehazing, High Efficiency Video Coding (HEVC) loop filtering and super-resolution, and computer vision for an autonomous robot. Our technical contributions are threefold: First, we propose to reformulate haze as a signal-dependent noise which allows us to uncover it by learning a structural residual. Based on our novel reformulation, we solve dehazing with recursive deep residual network and generative adversarial network which emphasizes on objective and perceptual quality, respectively. Second, we replace traditional filters in HEVC with a Convolutional Neural Network (CNN) filter. We show that our CNN filter could achieve 7% BD-rate saving when compared with traditional filters such as bilateral and deblocking filter. We also propose to incorporate a multi-scale CNN super-resolution module into HEVC. Such post-processing module could improve visual quality under extremely low bandwidth. Third, a transfer learning technique is implemented to support vision and autonomous decision making of a precision pollination robot. Good experimental results are reported with real-world data

An original framework for understanding human actions and body language by using deep neural networks

The evolution of both fields of Computer Vision (CV) and Artificial Neural Networks (ANNs) has allowed the development of efficient automatic systems for the analysis of people's behaviour. By studying hand movements it is possible to recognize gestures, often used by people to communicate information in a non-verbal way. These gestures can also be used to control or interact with devices without physically touching them. In particular, sign language and semaphoric hand gestures are the two foremost areas of interest due to their importance in Human-Human Communication (HHC) and Human-Computer Interaction (HCI), respectively. While the processing of body movements play a key role in the action recognition and affective computing fields. The former is essential to understand how people act in an environment, while the latter tries to interpret people's emotions based on their poses and movements; both are essential tasks in many computer vision applications, including event recognition, and video surveillance. In this Ph.D. thesis, an original framework for understanding Actions and body language is presented. The framework is composed of three main modules: in the first one, a Long Short Term Memory Recurrent Neural Networks (LSTM-RNNs) based method for the Recognition of Sign Language and Semaphoric Hand Gestures is proposed; the second module presents a solution based on 2D skeleton and two-branch stacked LSTM-RNNs for action recognition in video sequences; finally, in the last module, a solution for basic non-acted emotion recognition by using 3D skeleton and Deep Neural Networks (DNNs) is provided. The performances of RNN-LSTMs are explored in depth, due to their ability to model the long term contextual information of temporal sequences, making them suitable for analysing body movements. All the modules were tested by using challenging datasets, well known in the state of the art, showing remarkable results compared to the current literature methods