Toward the biological model of the hippocampus as the successor representation agent

The hippocampus is an essential brain region for spatial memory and learning. Recently, a theoretical model of the hippocampus based on temporal difference (TD) learning has been published. Inspired by the successor representation (SR) learning algorithms, which decompose value function of TD learning into reward and state transition, they argued that the rate of firing of CA1 place cells in the hippocampus represents the probability of state transition. This theory, called predictive map theory, claims that the hippocampus representing space learns the probability of transition from the current state to the future state. The neural correlates of expecting the future state are the firing rates of the CA1 place cells. This explanation is plausible for the results recorded in behavioral experiments, but it is lacking the neurobiological implications. Modifying the SR learning algorithm added biological implications to the predictive map theory. Similar with the simultaneous needs of information of the current and future state in the SR learning algorithm, the CA1 place cells receive two inputs from CA3 and entorhinal cortex. Mathematical transformation showed that the SR learning algorithm is equivalent to the heterosynaptic plasticity rule. The heterosynaptic plasticity phenomena in CA1 were discussed and compared with the modified SR update rule. This study attempted to interpret the TD algorithm as the neurobiological mechanism occurring in place learning, and to integrate the neuroscience and artificial intelligence approaches in the field.Comment: 9 pages, 1 figur

The effect of synaptic weight initialization in feature-based successor representation learning

After discovering place cells, the idea of the hippocampal (HPC) function to represent geometric spaces has been extended to predictions, imaginations, and conceptual cognitive maps. Recent research arguing that the HPC represents a predictive map; and it has shown that the HPC predicts visits to specific locations. This predictive map theory is based on successor representation (SR) from reinforcement learning. Feature-based SR (SF), which uses a neural network as a function approximation to learn SR, seems more plausible neurobiological model. However, it is not well known how different methods of weight (W) initialization affect SF learning. In this study, SF learners were exposed to simple maze environments to analyze SF learning efficiency and W patterns pattern changes. Three kinds of W initialization pattern were used: identity matrix, zero matrix, and small random matrix. The SF learner initiated with random weight matrix showed better performance than other three RL agents. We will discuss the neurobiological meaning of SF weight matrix. Through this approach, this paper tried to increase our understanding of intelligence from neuroscientific and artificial intelligence perspective.Comment: 11 pages, 8 figures including 2 supplementary figure

Insights from Analysis of Video Streaming Data to Improve Resource Management

Today a large portion of Internet traffic is video. Over The Top (OTT) service providers offer video streaming services by creating a large distributed cloud network on top of a physical infrastructure owned by multiple entities. Our study explores insights from video streaming activity by analyzing data collected from Korea's largest OTT service provider. Our analysis of nationwide data shows interesting characteristics of video streaming such as correlation between user profile information (e.g., age, sex) and viewing habits, viewing habits of users (when do the users watch? using which devices?), viewing patterns (early leaving viewer vs. steady viewer), etc. Video on Demand (VoD) streaming involves costly (and often limited) compute, storage, and network resources. Findings from our study will be beneficial for OTTs, Content Delivery Networks (CDNs), Internet Service Providers (ISPs), and Carrier Network Operators, to improve their resource allocation and management techniques.Comment: This is a preprint electronic version of the article accepted to IEEE CloudNet 201

Fabrication and Evaluation of Magnetic Micro actuators for Implantable Self-Clearing Glaucoma Drainage Devices

According to the World Health Organization, glaucoma is the second leading cause of blindness in the world. It currently affects more than 2.7 million people in the United States alone and over 79.6 million people worldwide are estimated to be inflicted by this debilitating disease by 2020. Glaucoma patients are often characterized with elevated intraocular pressure (IOP) and are treated with implantation of glaucoma drainage devices (GDD) to maintain optimum IOP. Although initially effective at delaying glaucoma progression, contemporary GDD often lead to numerous complications and only 50% of implanted devices remain functional after 5 years. Biofouling is seen to be one of the leading cause for the failure of GDDs. In order to overcome biological blockage, we propose a self-clearing glaucoma drainage device using integrated magnetic microactuators. Here we report on the maskless photolithographic fabrication results of magnetic microactuators to be integrated into a bespoke GDD. The maskless photography enabled rapid prototyping of microdevices using low-cost materials in contrast to conventional lithographic methods. The fabricated devices were able to produce maximum deflection of 57.9 degree at magnetic field strength of 32.6 kA/m. The static response of the fabricated devices was compared with the theoretical data

Regularizing Towards Soft Equivariance Under Mixed Symmetries

Datasets often have their intrinsic symmetries, and particular deep-learning models called equivariant or invariant models have been developed to exploit these symmetries. However, if some or all of these symmetries are only approximate, which frequently happens in practice, these models may be suboptimal due to the architectural restrictions imposed on them. We tackle this issue of approximate symmetries in a setup where symmetries are mixed, i.e., they are symmetries of not single but multiple different types and the degree of approximation varies across these types. Instead of proposing a new architectural restriction as in most of the previous approaches, we present a regularizer-based method for building a model for a dataset with mixed approximate symmetries. The key component of our method is what we call equivariance regularizer for a given type of symmetries, which measures how much a model is equivariant with respect to the symmetries of the type. Our method is trained with these regularizers, one per each symmetry type, and the strength of the regularizers is automatically tuned during training, leading to the discovery of the approximation levels of some candidate symmetry types without explicit supervision. Using synthetic function approximation and motion forecasting tasks, we demonstrate that our method achieves better accuracy than prior approaches while discovering the approximate symmetry levels correctly.Comment: Proceedings of the International Conference on Machine Learning (ICML), 202

Intense ultraviolet emission from needle-like WO3 nanostructures synthesized by noncatalytic thermal evaporation

Photoluminescence measurements showed that needle-like tungsten oxide nanostructures synthesized at 590°C to 750°C by the thermal evaporation of WO3 nanopowders without the use of a catalyst had an intense near-ultraviolet (NUV) emission band that was different from that of the tungsten oxide nanostructures obtained in other temperature ranges. The intense NUV emission might be due to the localized states associated with oxygen vacancies and surface states

Probabilistic Imputation for Time-series Classification with Missing Data

Multivariate time series data for real-world applications typically contain a significant amount of missing values. The dominant approach for classification with such missing values is to impute them heuristically with specific values (zero, mean, values of adjacent time-steps) or learnable parameters. However, these simple strategies do not take the data generative process into account, and more importantly, do not effectively capture the uncertainty in prediction due to the multiple possibilities for the missing values. In this paper, we propose a novel probabilistic framework for classification with multivariate time series data with missing values. Our model consists of two parts; a deep generative model for missing value imputation and a classifier. Extending the existing deep generative models to better capture structures of time-series data, our deep generative model part is trained to impute the missing values in multiple plausible ways, effectively modeling the uncertainty of the imputation. The classifier part takes the time series data along with the imputed missing values and classifies signals, and is trained to capture the predictive uncertainty due to the multiple possibilities of imputations. Importantly, we show that na\"ively combining the generative model and the classifier could result in trivial solutions where the generative model does not produce meaningful imputations. To resolve this, we present a novel regularization technique that can promote the model to produce useful imputation values that help classification. Through extensive experiments on real-world time series data with missing values, we demonstrate the effectiveness of our method