On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models

This paper addresses the general problem of reinforcement learning (RL) in partially observable environments. In 2013, our large RL recurrent neural networks (RNNs) learned from scratch to drive simulated cars from high-dimensional video input. However, real brains are more powerful in many ways. In particular, they learn a predictive model of their initially unknown environment, and somehow use it for abstract (e.g., hierarchical) planning and reasoning. Guided by algorithmic information theory, we describe RNN-based AIs (RNNAIs) designed to do the same. Such an RNNAI can be trained on never-ending sequences of tasks, some of them provided by the user, others invented by the RNNAI itself in a curious, playful fashion, to improve its RNN-based world model. Unlike our previous model-building RNN-based RL machines dating back to 1990, the RNNAI learns to actively query its model for abstract reasoning and planning and decision making, essentially "learning to think." The basic ideas of this report can be applied to many other cases where one RNN-like system exploits the algorithmic information content of another. They are taken from a grant proposal submitted in Fall 2014, and also explain concepts such as "mirror neurons." Experimental results will be described in separate papers.Comment: 36 pages, 1 figure. arXiv admin note: substantial text overlap with arXiv:1404.782

Supervised learning with artificial hydrocarbon networks: an open source implementation and its applications

Artificial hydrocarbon networks (AHN) is a novel supervised learning method inspired on the structure and the inner chemical mechanisms of organic compounds. As any other cutting-edge algorithm, there are two challenges to be faced: time-consuming for encoding and complications to connect with other technologies. Large and open source platforms have proved to be an alternative solution to the latter challenges. In that sense, this paper aims to introduce the ahnr package for R that implements AHN. It provides several functions to create, train, test and visualize AHN. It also includes conventional functions to easily interact with the trained models. For illustration purposes, it presents several examples about the applications of AHN in engineering, as well as, the way to use it. This package is intended to be very useful for scientists and applied researchers interested in machine learning and data modeling. Package availability is in the Comprehensive R Archive Network.Comment: 13 pages, 8 figure

The role of behavior modifiers in representation development

We address the problem of the development of representations and their relationship to the environment. We study a software agent which develops in a network a representation of its simple environment which captures and integrates the relationships between agent and environment through a closure mechanism. The inclusion of a variable behavior modifier allows better representation development. This can be confirmed with an internal description of the closure mechanism, and with an external description of the properties of the representation network.Comment: 8 page

The State of the Art Recognize in Arabic Script through Combination of Online and Offline

Handwriting recognition refers to the identification of written characters. Handwriting recognition has become an acute research area in recent years for the ease of access of computer science. In this paper primarily discussed On-line and Off-line handwriting recognition methods for Arabic words which are often used among then across the Middle East and North Africa People. Arabic word online handwriting recognition is a very challenging task due to its cursive nature. Because of the characteristic of the whole body of the Arabic script, namely connectivity between the characters, thereby the segmentation of An Arabic script is very difficult. In this paper we introduced an Arabic script multiple classifier system for recognizing notes written on a Starboard. This Arabic script multiple classifier system combines one off-line and on-line handwriting recognition systems. The Arabic script recognizers are all based on Hidden Markov Models but vary in the way of preprocessing and normalization. To combine the Arabic script output sequences of the recognizers, we incrementally align the word sequences using a norm string matching algorithm. The Arabic script combination we could increase the system performance over the excellent character recognizer by about 3%. The proposed technique is also the necessary step towards character recognition, person identification, personality determination where input data is processed from all perspectives.Comment: Pages 7, Figure 6, Table 2. arXiv admin note: text overlap with arXiv:1110.1488 by other author

Representation Learning for Dynamic Graphs: A Survey

Graphs arise naturally in many real-world applications including social networks, recommender systems, ontologies, biology, and computational finance. Traditionally, machine learning models for graphs have been mostly designed for static graphs. However, many applications involve evolving graphs. This introduces important challenges for learning and inference since nodes, attributes, and edges change over time. In this survey, we review the recent advances in representation learning for dynamic graphs, including dynamic knowledge graphs. We describe existing models from an encoder-decoder perspective, categorize these encoders and decoders based on the techniques they employ, and analyze the approaches in each category. We also review several prominent applications and widely used datasets and highlight directions for future research.Comment: Accepted at JMLR, 73 pages, 2 figure

A Novel Hybrid Crossover based Artificial Bee Colony Algorithm for Optimization Problem

Artificial bee colony (ABC) algorithm has proved its importance in solving a number of problems including engineering optimization problems. ABC algorithm is one of the most popular and youngest member of the family of population based nature inspired meta-heuristic swarm intelligence method. ABC has been proved its superiority over some other Nature Inspired Algorithms (NIA) when applied for both benchmark functions and real world problems. The performance of search process of ABC depends on a random value which tries to balance exploration and exploitation phase. In order to increase the performance it is required to balance the exploration of search space and exploitation of optimal solution of the ABC. This paper outlines a new hybrid of ABC algorithm with Genetic Algorithm. The proposed method integrates crossover operation from Genetic Algorithm (GA) with original ABC algorithm. The proposed method is named as Crossover based ABC (CbABC). The CbABC strengthens the exploitation phase of ABC as crossover enhances exploration of search space. The CbABC tested over four standard benchmark functions and a popular continuous optimization problem

Study Of E-Smooth Support Vector Regression And Comparison With E- Support Vector Regression And Potential Support Vector Machines For Prediction For The Antitubercular Activity Of Oxazolines And Oxazoles Derivatives

A new smoothing method for solving ? -support vector regression (?-SVR), tolerating a small error in fitting a given data sets nonlinearly is proposed in this study. Which is a smooth unconstrained optimization reformulation of the traditional linear programming associated with a ?-insensitive support vector regression. We term this redeveloped problem as ?-smooth support vector regression (?-SSVR). The performance and predictive ability of ?-SSVR are investigated and compared with other methods such as LIBSVM (?-SVR) and P-SVM methods. In the present study, two Oxazolines and Oxazoles molecular descriptor data sets were evaluated. We demonstrate the merits of our algorithm in a series of experiments. Primary experimental results illustrate that our proposed approach improves the regression performance and the learning efficiency. In both studied cases, the predictive ability of the ?- SSVR model is comparable or superior to those obtained by LIBSVM and P-SVM. The results indicate that ?-SSVR can be used as an alternative powerful modeling method for regression studies. The experimental results show that the presented algorithm ?-SSVR, plays better precisely and effectively than LIBSVMand P-SVM in predicting antitubercular activity

A Survey and Critique of Multiagent Deep Reinforcement Learning

Deep reinforcement learning (RL) has achieved outstanding results in recent years. This has led to a dramatic increase in the number of applications and methods. Recent works have explored learning beyond single-agent scenarios and have considered multiagent learning (MAL) scenarios. Initial results report successes in complex multiagent domains, although there are several challenges to be addressed. The primary goal of this article is to provide a clear overview of current multiagent deep reinforcement learning (MDRL) literature. Additionally, we complement the overview with a broader analysis: (i) we revisit previous key components, originally presented in MAL and RL, and highlight how they have been adapted to multiagent deep reinforcement learning settings. (ii) We provide general guidelines to new practitioners in the area: describing lessons learned from MDRL works, pointing to recent benchmarks, and outlining open avenues of research. (iii) We take a more critical tone raising practical challenges of MDRL (e.g., implementation and computational demands). We expect this article will help unify and motivate future research to take advantage of the abundant literature that exists (e.g., RL and MAL) in a joint effort to promote fruitful research in the multiagent community.Comment: Under review since Oct 2018. Earlier versions of this work had the title: "Is multiagent deep reinforcement learning the answer or the question? A brief survey

Architecture for Pseudo Acausal Evolvable Embedded Systems

Advances in semiconductor technology are contributing to the increasing complexity in the design of embedded systems. Architectures with novel techniques such as evolvable nature and autonomous behavior have engrossed lot of attention. This paper demonstrates conceptually evolvable embedded systems can be characterized basing on acausal nature. It is noted that in acausal systems, future input needs to be known, here we make a mechanism such that the system predicts the future inputs and exhibits pseudo acausal nature. An embedded system that uses theoretical framework of acausality is proposed. Our method aims at a novel architecture that features the hardware evolability and autonomous behavior alongside pseudo acausality. Various aspects of this architecture are discussed in detail along with the limitations.Comment: 4 pages, 2 figures. Submitted to SASO 200

Scope of Research on Particle Swarm Optimization Based Data Clustering

Optimization is nothing but a mathematical technique which finds maxima or minima of any function of concern in some realistic region. Different optimization techniques are proposed which are competing for the best solution. Particle Swarm Optimization (PSO) is a new, advanced, and most powerful optimization methodology that performs empirically well on several optimization problems. It is the extensively used Swarm Intelligence (SI) inspired optimization algorithm used for finding the global optimal solution in a multifaceted search region. Data clustering is one of the challenging real world applications that invite the eminent research works in variety of fields. Applicability of different PSO variants to data clustering is studied in the literature, and the analyzed research work shows that, PSO variants give poor results for multidimensional data. This paper describes the different challenges associated with multidimensional data clustering and scope of research on optimizing the clustering problems using PSO. We also propose a strategy to use hybrid PSO variant for clustering multidimensional numerical, text and image data.Comment: 7 pages, 6 figures, 1 table, published with International Journal of Computer Science Trends and Technology (IJCST