A Biologically-Inspired Dual Stream World Model

The medial temporal lobe (MTL), a brain region containing the hippocampus and nearby areas, is hypothesized to be an experience-construction system in mammals, supporting both recall and imagination of temporally-extended sequences of events. Such capabilities are also core to many recently proposed ``world models" in the field of AI research. Taking inspiration from this connection, we propose a novel variant, the Dual Stream World Model (DSWM), which learns from high-dimensional observations and dissociates them into context and content streams. DSWM can reliably generate imagined trajectories in novel 2D environments after only a single exposure, outperforming a standard world model. DSWM also learns latent representations which bear a strong resemblance to place cells found in the hippocampus. We show that this representation is useful as a reinforcement learning basis function, and that the generative model can be used to aid the policy learning process using Dyna-like updates

A Comparison of Wholesaler/Retailer Business Characteristics of Natural Products between Ghana and Rwanda

The usage of natural products is becoming an increasingly common consumer phenomenon due to increasing health consciousness, and because of their naturalness, and medicinal qualities of the products. African countries are very rich with natural products resources and supplies. The continent’s rich botanical heritage offers an excellent opportunity to diversify away from other traditional exports. Europe and the USA are particularly promising markets for natural products. Thus, it is advantageous to examine development of natural products exporting as alternative or complimentary economic opportunities for many African people, especially those in the rural areas. This paper has explores both factors which promote and which act as obstacles to the natural products market, specifically in the retail and wholesale portions of the value chain in Ghana and Rwanda.Agribusiness,

An Overview of Marketing of Ghana Natural Products

The study finds strong correlations between natural products business performance and the impeding factors. The impediments include access to finance and markets, lack of herbal market information especially relating to external markets. Additionally, there is lack of processing capacity, while at the same time most if not all the natural products business operators lack technical training relating to product handling. However, there is big potential for success, the top ten traded natural products, may be exploited initially, both domestically and for export market, given range of perceived use. The constraints identified require concerted efforts from all stakeholders to recognize the importance of this sub-sector in providing opportunities to successful development.Marketing,

Strategies for Using Proximal Policy Optimization in Mobile Puzzle Games

While traditionally a labour intensive task, the testing of game content is progressively becoming more automated. Among the many directions in which this automation is taking shape, automatic play-testing is one of the most promising thanks also to advancements of many supervised and reinforcement learning (RL) algorithms. However these type of algorithms, while extremely powerful, often suffer in production environments due to issues with reliability and transparency in their training and usage. In this research work we are investigating and evaluating strategies to apply the popular RL method Proximal Policy Optimization (PPO) in a casual mobile puzzle game with a specific focus on improving its reliability in training and generalization during game playing. We have implemented and tested a number of different strategies against a real-world mobile puzzle game (Lily's Garden from Tactile Games). We isolated the conditions that lead to a failure in either training or generalization during testing and we identified a few strategies to ensure a more stable behaviour of the algorithm in this game genre.Comment: 10 pages, 8 figures, to be published in 2020 Foundations of Digital Games conferenc

Systems Biology-Based Analysis Indicates Global Transcriptional Impairment in Lead-Treated Human Neural Progenitor Cells

Lead poisoning effects are wide and include nervous system impairment, peculiarly during development, leading to neural damage. Lead interaction with calcium and zinc-containing metalloproteins broadly affects cellular metabolism since these proteins are related to intracellular ion balance, activation of signaling transduction cascades, and gene expression regulation. In spite of lead being recognized as a neurotoxin, there are gaps in knowledge about the global effect of lead in modulating the transcription of entire cellular systems in neural cells. In order to investigate the effects of lead poisoning in a systemic perspective, we applied the transcriptogram methodology in an RNA-seq dataset of human embryonic-derived neural progenitor cells (ES-NP cells) treated with 30 µM lead acetate for 26 days. We observed early downregulation of several cellular systems involved with cell differentiation, such as cytoskeleton organization, RNA, and protein biosynthesis. The downregulated cellular systems presented big and tightly connected networks. For long treatment times (12 to 26 days), it was possible to observe a massive impairment in cell transcription profile. Taking the enriched terms together, we observed interference in all layers of gene expression regulation, from chromatin remodeling to vesicle transport. Considering that ES-NP cells are progenitor cells that can originate other neural cell types, our results suggest that lead-induced gene expression disturbance might impair cells’ ability to differentiate, therefore influencing ES-NP cells’ fate

Obstacle tower : a generalization challenge in vision, control, and planning

The rapid pace of recent research in AI has been driven in part by the presence of fast and challenging simulation environments. These environments often take the form of games; with tasks ranging from simple board games, to competitive video games. We propose a new benchmark - Obstacle Tower: a high fidelity, 3D, 3rd person, procedurally generated environment. An agent playing Obstacle Tower must learn to solve both low-level control and high-level planning problems in tandem while learning from pixels and a sparse reward signal. Unlike other benchmarks such as the Arcade Learning Environment, evaluation of agent performance in Obstacle Tower is based on an agent's ability to perform well on unseen instances of the environment. In this paper we outline the environment and provide a set of baseline results produced by current state-of-the-art Deep RL methods as well as human players. These algorithms fail to produce agents capable of performing near human level.peer-reviewe

Learning and Acting with Predictive Cognitive Maps

Humans and other mammals possess two remarkable abilities: the capacity to store and retrieve a seemingly boundless series of episodic memories, and the capacity to quickly make sense of and navigate their changing environments. The latter has been described as a cognitive map, and along with the capacity to store and retrieve narrative memories, has been largely localized to the medial temporal lobe. Recent theorists have suggested that these two capacities are both aspects of a single unified system of ‘experience construction.’ In such a system, complex high-dimensional sensory experiences represented in the cortex are indexed by a low-dimensional representation within the medial temporal lobe. The dynamics of this representation then allow for the generation of coherent sequences of activation which correspond to coherent narrative experiences, as well as coherent trajectories through the environment, supporting both memory and navigation. Such a theoretical perspective bears a strong resemblance to a recent class of deep neural networks called generative temporal models. In this work we explore this connection by introducing a series of increasingly complex generative temporal models, and analyzing each of their properties. We find that these models are able to learn representations which bear a strong resemblance to known representations within the medial temporal lobe, such as place and time cells. Furthermore, we demonstrate that these representations are useful for rapidly learning to perform downstream goal-directed navigation tasks using biologically plausible reinforcement learning rules. We also examine the ways in which these models can be extended to display adaptation to changes in the structure or content of the environment, a key property of the cognitive map. Finally, we compare the behavior of artificial agents utilizing these learned representations to those of humans in a complex virtual navigation task. In doing so, we find evidence that humans utilize a hybrid behavioral strategy, and that such a strategy can be modeled by artificial agents utilizing a learned place cell like representation

Learning and Acting with Predictive Cognitive Maps

Humans and other mammals possess two remarkable abilities: the capacity to store and retrieve a seemingly boundless series of episodic memories, and the capacity to quickly make sense of and navigate their changing environments. The latter has been described as a cognitive map, and along with the capacity to store and retrieve narrative memories, has been largely localized to the medial temporal lobe. Recent theorists have suggested that these two capacities are both aspects of a single unified system of ‘experience construction.’ In such a system, complex high-dimensional sensory experiences represented in the cortex are indexed by a low-dimensional representation within the medial temporal lobe. The dynamics of this representation then allow for the generation of coherent sequences of activation which correspond to coherent narrative experiences, as well as coherent trajectories through the environment, supporting both memory and navigation. Such a theoretical perspective bears a strong resemblance to a recent class of deep neural networks called generative temporal models. In this work we explore this connection by introducing a series of increasingly complex generative temporal models, and analyzing each of their properties. We find that these models are able to learn representations which bear a strong resemblance to known representations within the medial temporal lobe, such as place and time cells. Furthermore, we demonstrate that these representations are useful for rapidly learning to perform downstream goal-directed navigation tasks using biologically plausible reinforcement learning rules. We also examine the ways in which these models can be extended to display adaptation to changes in the structure or content of the environment, a key property of the cognitive map. Finally, we compare the behavior of artificial agents utilizing these learned representations to those of humans in a complex virtual navigation task. In doing so, we find evidence that humans utilize a hybrid behavioral strategy, and that such a strategy can be modeled by artificial agents utilizing a learned place cell like representation