ANALYSIS OF CELLULAR DYNAMIC BINARY NEURAL NETWORKS

This paper studies dynamic binary neural networks characterized by signum activation function, local connection parameters and integer threshold paremeters. The DBNN is constructed by applying delayed feedback to the binary neural networks. The network can generate various periodic orbits. The dynamics is simplified into a digital return map on a set of lattice points. We analyze the dynamics by replacing The DBNN with a simple class network in this paper. We consider the relationship between cellular automata and DBNN. Calculating feature quantities, we investigate the relationship between a simple class of CA and DBNN with local connection. Analysis of the DBNN is important not only as fundamental nonlinear problems but also for engineering applications

ANALYSIS OF SPARSE DYNAMIC BINARY NEURAL NETWORKS AND ITS APPLICATIONS

This paper studies hardware implementation of a simple dynamic binary neural network. The dynamic binary neural network is a simple two-layer network with a delayed feedback and that can generate various periodic orbits. The network is characterized by local binary connection and signum activation function. First, using an FPGA board, a test circuit is implemented. The signum activation function is realized by a majority decision circuit and the binary connection is realized by switches and inverters. The circuit operation is confirmed experimentally

CONNECTION SPARSITY AND ORBIT STABILITY IN DYNAMIC BINARY NEURAL NETWORKS

Dynamic binary neural networks are recurrent type neural networks characterized by ternary connection parameters and signum activate function. Depending on the parameters, the network can generate various binary periodic orbits. The ternary connection parameters and signum activation function are suitable for precise analysis and hardware implementation. First, we investigate influence of connection sparsity on stability of a periodic orbit. As the sparsity increases, stability of a periodic orbit tends to be reinforced. As the sparsity increases further, stability tends to be weakened and various transient phenomena exist

Analysis of dynamic binary neural networks based on evolutionary computation

We consider evolutionary synthesis of permutation binary neural network characterized by global permutation connection and local binary connection. The network is simple version of dynamic binary neural network. Although the networks are simple and the parameter space is small, the networks exhibit various periodic orbits of binary vectors. Since analysis of all the periodic orbits is not easy, we focus on globally stable periodic orbits such that almost all initial points fall into the orbits. We present a simple search algorithm for globally stable periodic orbits. Applying the algorithm, we have clarified that permutation binary neural networks generate a variety of globally stable periodic orbits

Analysis of various steady states and transient phenomena in digital maps : foundation for theory construction and engineering applications

研究成果の概要 (和文) : デジタルマップ(Dmap)の解析と実装に関して以下のような成果を得た。まず、周期軌道の豊富さと安定性に関する特徴量を用いた解析法を考案し、典型例を解析し、現象の基本的な分類を行った。次に、簡素な進化計算によって所望のDmapを合成するアルゴリズムを考案した。アルゴリズムの個体はDmapに対応し、個体数は柔軟に変化する。典型的な例題によってアルゴリズムの妥当性を確認した。さらに、Dmapをデジタルスパイキングニューロン(DSN)によって実現する方法を構築した。DSNは2つのシフトレジスタと配線回路で構成され、様々なスパイク列を生成する。FPGAによる簡素な試作回路を構成し、動作を確認した。研究成果の概要 (英文) : We have studied analysis and implementation of digital maps (Dmaps). The major results are as the following. First, we have developed an analysis method based on two feature quantities. The first quantity characterizes plentifulness of periodic orbits and the second quantity characterizes stability of the periodic orbits. Applying the method, typical Dmap examples are analyzed and basic phenomena are classified. Second, we have developed a simple evolutionary algorithm to realize a desired Dmap. The algorithm uses individuals each of which corresponds to one Dmap and the number of individuals can vary flexibly. Using typical example problems, the algorithm efficiency is confirmed. Third, we have developed a realization method of Dmaps by means of digital spiking neurons (DSNs). The DSN consists of two shift registers connected by a wiring circuit and can generate various periodic spike-trains. Presenting a FPGA based simple test circuit, the DSN dynamics is confirmed

TRANSITION FROM FIXED POINTS TO PERIODIC ORBITS IN DYNAMIC BINARY NEURAL NETWORKS

A dynamic binary neural network is a simple two-layer network with a delayed feedback. The network is characterized by signum activation function parameters, connection parameters, and integer threshold parameters. Depending on the parameters and initial value, the network can generate various binary periodic orbits and fixed point. The dynamics is represented by a digital return map on a set of lattice points. we considered transition from a target basic periodic orbit to a set of fixed points and a target basic periodic orbit to a periodic orbit with period 2. Performing elementary numerical experiments, we have found two typical patterns

ANALYSIS AND APPLICATIONS OF BINARY PERIODIC ORBITS IN DYNAMIC BINARY NEURAL NETWORKS

This paper studies basic dynamics of simple dynamic binary neural networks and their applications. The network is characterized by local binary connection and signum activation function. Depending on the parameters and initial condition, the network can generate various binary periodic orbits. The binary connection is suitable for FPGA based hardware implementation. We consider two target periodic orbits based on the insect walking gaits and switching of them. Implementing a test circuit on the Verilog, switching of the periodic orbits is confirmed experimentally. These results will be developed into applications to central pattern generators

SYNTHESIS OF DIGITAL SPIKE MAPS BY THE EVOLUTIONARY ALGORITHM

This paper studies analysis and implement of Digital Dpike Maps(DSM). In order to visualize dynamics of spike-trains, we introduce the DSM. The DSM is a digital version of analog one-dimensional maps. The Dmap is related to various digital dynamical systems including cellular automata and dynamic binary neural networks, digital spikeing neuron. In order to realization desired DSM, we present a simple evolutionary algorithm. It is a type of optimization algorithm inspired by biological mechanisms

APPLICATION OF EVOLUTIONARY MULTI-OBJECTIVE OPTIMIZATION ALGORITHM TO DYNAMIC BINARY NEURAL NETWORKS

This paper studies application of an evolutionary multi-objective optimization algorithm. A dynamic binary neural networks is characterized by the signum activation function and ternary connection parameters. Depending on the connection parameters, this network can generate various binary periodic orbits. In order to evaluate the performance, we consider the bi-objective problem corresponding to stability of the binary periodic orbits and sparsity of the connection parameters. Although uni-objective optimization problems require the optimization of only one objective, multi-objective optimization problems require the simultaneous optimization of multiple objectives. In order to optimize the bi-objective problem, we present a multiobjective evolutionary algorithm based on decomposition. This algorithm decomposes the bi-objective problem into multiple subproblems and can optimize the problem effectively. Performing elementary numerical experiments for typical examples of binary periodic orbits, it is confirmed that the algorithm realizes both strong orbit stability and appropriate connection sparsity. It is also confirmed that the algorithm outperforms another algorithm based on the Lasso regularization

Analysis and learning of dynamic binary neural networks which can generate variable phenomena

研究成果の概要 (和文) : 2層の動的バイナリーニューラルネット(DBNN)に2値周期軌道(BPO)を銘記する学習法を構築した。同手法をパワーエレクトロニクスの基本回路の制御信号に対応する教師信号BPOに適用し、手法の有効性を確認した。また、デジタルリターンマップを用いてDBNNの動作を視覚化する方法も提案し、学習過程の把握に有効であることを明らかにした。DBNNにBPOが銘記できた場合に、結合行列をスパース化すると、それに落ち込む初期値の数が増え、安定性が強化される場合のあることを示した。いくつかの基本的な例題教師信号によって、そのスパース化の有効性を確認した。研究成果の概要 (英文) : We have constructed a learning method to store one desired binary periodic orbit (BPO) into to the dynamic binary neural networks is presented. Applying the method to teacher signal BPOs that correspond to control signals of basic switching power converters, the efficiency of the method is confirmed. Introducing a digital return map, the dynamics of the DBNN is visualized and analyzed.In the case where a desired BPO can be stored into a DBNN, we have clarified that　stability of the stored BPO can be reinforced (the number of initial points falling into the BPO is increased) by sparsifying connection matrix. In several basic examples of teacher signals, the efficiency of the sparsification is confirmed