Circuit Design of Multimodal Attention Memristive Network for Affective Video Content Analysis

Affective video content analysis aims at automatically identifying human emotion triggered by video, which plays an important role in mental health monitoring. This paper proposes a multimodal attention memristive network for affective video content analysis, which offers an energy-efficient approach with low time consumption and high classification accuracy. To illustrate the complexity of the proposed multimodal attention memristive network, two core modules are proposed. Firstly, unimodal feature representation module with cascaded configuration is designed to capture unique characteristics from multimodal signals. Then, multimodal local-global fusion module is proposed to stimulate the process of multimodal information sensing and processing in human brain. Furthermore, the proposed system is validated by applying it to affective content analysis. The experimental results demonstrate that the multimodal attention memristive network outperforms the existing state-of-the-art methods with high classification accuracy and low time consumption.10.13039/501100001809-National Natural Science Foundation of China (Grant Number: 62001149

Adaptation Accelerating Sampling-based Bayesian Inference in Attractor Neural Networks

The brain performs probabilistic Bayesian inference to interpret the external world. The sampling-based view assumes that the brain represents the stimulus posterior distribution via samples of stochastic neuronal responses. Although the idea of sampling-based inference is appealing, it faces a critical challenge of whether stochastic sampling is fast enough to match the rapid computation of the brain. In this study, we explore how latent feature sampling can be accelerated in neural circuits. Specifically, we consider a canonical neural circuit model called continuous attractor neural networks (CANNs) and investigate how sampling-based inference of latent continuous variables is accelerated in CANNs. Intriguingly, we find that by including noisy adaptation in the neuronal dynamics, the CANN is able to speed up the sampling process significantly. We theoretically derive that the CANN with noisy adaptation implements the efficient sampling method called Hamiltonian dynamics with friction, where noisy adaption effectively plays the role of momentum. We theoretically analyze the sampling performances of the network and derive the condition when the acceleration has the maximum effect. Simulation results validate our theoretical analyses. We further extend the model to coupled CANNs and demonstrate that noisy adaptation accelerates the sampling of the posterior distribution of multivariate stimuli. We hope that this study enhances our understanding of how Bayesian inference is realized in the brain

A Brain-Inspired In-Memory Computing System for Neuronal Communication via Memristive Circuits

This work was supported in part by the National Natural Science Foundation of China under Grant U1909201 and Grant 62001149, and the Natural Science Foundation of Zhejiang Province under Grant LQ21F010009

Design and Implementation of a Flexible Neuromorphic Computing System for Affective Communication via Memristive Circuits

National Natural Science Foundation of China under Grant 62001149, Natural Science Foundation of Zhejiang Province under Grant LQ21F010009 and Fundamental Research funds for the provincial Universities of Zhejiang under Grant GK229909299001-06

Circuit Design of Memristor-based GRU and its Applications in SOC Estimation

10.13039/501100001809-National Natural Science Foundation (Grant Number: 62001149); Zhejiang Provincial Nature Science Foundation of China under Grant No. LQ21F010009; Fundamental Research Funds for the Provincial Universities of Zhejiang under Grant No. GK229909299001-06

On mining latent treatment patterns from electronic medical records

Clinical pathway (CP) analysis plays an important role in health-care management in ensuring specialized, standardized, normalized and sophisticated therapy procedures for individual patients. Recently, with the rapid development of hospital information systems, a large volume of electronic medical records (EMRs) has been produced, which provides a comprehensive source for CP analysis. In this paper, we are concerned with the problem of utilizing the heterogeneous EMRs to assist CP analysis and improvement. More specifically, we develop a probabilistic topic model to link patient features and treatment behaviors together to mine treatment patterns hidden in EMRs. Discovered treatment patterns, as actionable knowledge representing the best practice for most patients in most time of their treatment processes, form the backbone of CPs, and can be exploited to help physicians better understand their specialty and learn from previous experiences for CP analysis and improvement. Experimental results on a real collection of 985 EMRs collected from a Chinese hospital show that the proposed approach can effectively identify meaningful treatment patterns from EMRs

A physics-oriented memristor model with the coexistence of NDR effect and RS memory behavior for bio-inspired computing

Bio-inspired computing promises fundamentally different ways to advances in artificial intelligence with extreme energy efficiency. Memristive technologies due to the non-volatility, high density, low-power, and synaptic bionic properties can help in realizing bio-inspired architecture and its hardware implementation. This paper proposes a novel physics-oriented memristor model with coexistence of negative differential resistance (NDR) effect and resistive switching (RS) memory behavior for bio-inspired computing. Firstly, an Ag/TiOx/FTO memristor is fabricated using sol-gel and magnetron sputtering method, and its performance test demonstrates that the coexistence of NDR effect and RS memory behavior can be modulated by the moisture. Then, a physical-oriented memristor model is constructed, which provides the possibility to explore the dynamics of the coexistence of NDR effect and RS memory behavior in simulation. Furthermore, a memristor-based affective computing circuit emulating the process of human affective associative learning is designed. The experiment demonstrates that the coexistence of NDR effect and RS memory behavior can change the memory time without additional circuit and cost, which is expected to realize the automatic conversion from short-term memory to long-term memory in bio-inspired computing.National Natural Science Foundation of China under Grant 62001149 and Natural Science Foundation of Zhejiang Province under Grant LQ21F010009

Memristive Circuit Design of Sequencer Network for Human Emotion Classification

Mental health problem is an increasingly common social issue leading to diseases such as depression, addiction, and heart attack. Facial expression is one of the most natural and universal signals for human beings to convey their emotional states and behavior intentions. Numerous studies have been conducted on automatic human emotion classification that can effectively establish the relationship between facial expression and mental health, while still suffer from intensive computation and low efficiency. Here, we present a memristive circuit design of Sequencer network for human emotion classification, which offers an environmentally friendly approach with low cost and easily deployable hardware. Specifically, a kind of eco-friendly memristor is fabricated using two-dimensional (2D) materials, and the corresponding testing performance is conducted to make sure its efficiency and stability. Then, the memristor-based Sequencer block, as a core component of Sequencer network, consisting of bidirectional long short-term memory (BiLSTM) circuit and some necessary function circuit modules is proposed. Based on this, the memristive Sequencer network can be achieved. Furthermore, the proposed memristive Sequencer network is applied for human emotion classification. The experimental results demonstrate that the proposed circuit has advantages in computational efficiency and cost, comparable to the main existing software-based methods.National Natural Science Foundation of China (grant no. 62001149) and the Natural Science Foundation of Zhejiang Province (grant no. LQ21F010009)

A Brain-inspired Hierarchical Interactive In-memory Computing System and its Application in Video Sentiment Analysis

Fundamental Research Funds for the Provincial University of Zhejiang (Grant Number: GK229909299001-06); 10.13039/501100001809-National Natural Science Foundation of China (Grant Number: 62001149); 10.13039/501100004731-Natural Science Foundation of Zhejiang Province (Grant Number: LQ21F010009)