A Broad Class of Discrete-Time Hypercomplex-Valued Hopfield Neural Networks

In this paper, we address the stability of a broad class of discrete-time hypercomplex-valued Hopfield-type neural networks. To ensure the neural networks belonging to this class always settle down at a stationary state, we introduce novel hypercomplex number systems referred to as real-part associative hypercomplex number systems. Real-part associative hypercomplex number systems generalize the well-known Cayley-Dickson algebras and real Clifford algebras and include the systems of real numbers, complex numbers, dual numbers, hyperbolic numbers, quaternions, tessarines, and octonions as particular instances. Apart from the novel hypercomplex number systems, we introduce a family of hypercomplex-valued activation functions called $\mathcal{B}$-projection functions. Broadly speaking, a $\mathcal{B}$-projection function projects the activation potential onto the set of all possible states of a hypercomplex-valued neuron. Using the theory presented in this paper, we confirm the stability analysis of several discrete-time hypercomplex-valued Hopfield-type neural networks from the literature. Moreover, we introduce and provide the stability analysis of a general class of Hopfield-type neural networks on Cayley-Dickson algebras

Learning Schemes for Recurrent Neural Networks

兵庫県立大学大学院202

Complex Neural Networks for Audio

Audio is represented in two mathematically equivalent ways: the real-valued time domain (i.e., waveform) and the complex-valued frequency domain (i.e., spectrum). There are advantages to the frequency-domain representation, e.g., the human auditory system is known to process sound in the frequency-domain. Furthermore, linear time-invariant systems are convolved with sources in the time-domain, whereas they may be factorized in the frequency-domain. Neural networks have become rather useful when applied to audio tasks such as machine listening and audio synthesis, which are related by their dependencies on high quality acoustic models. They ideally encapsulate fine-scale temporal structure, such as that encoded in the phase of frequency-domain audio, yet there are no authoritative deep learning methods for complex audio. This manuscript is dedicated to addressing the shortcoming. Chapter 2 motivates complex networks by their affinity with complex-domain audio, while Chapter 3 contributes methods for building and optimizing complex networks. We show that the naive implementation of Adam optimization is incorrect for complex random variables and show that selection of input and output representation has a significant impact on the performance of a complex network. Experimental results with novel complex neural architectures are provided in the second half of this manuscript. Chapter 4 introduces a complex model for binaural audio source localization. We show that, like humans, the complex model can generalize to different anatomical filters, which is important in the context of machine listening. The complex model\u27s performance is better than that of the real-valued models, as well as real- and complex-valued baselines. Chapter 5 proposes a two-stage method for speech enhancement. In the first stage, a complex-valued stochastic autoencoder projects complex vectors to a discrete space. In the second stage, long-term temporal dependencies are modeled in the discrete space. The autoencoder raises the performance ceiling for state of the art speech enhancement, but the dynamic enhancement model does not outperform other baselines. We discuss areas for improvement and note that the complex Adam optimizer improves training convergence over the naive implementation

Threshold Switching and Self-Oscillation in Niobium Oxide

Volatile threshold switching, or current controlled negative differential resistance (CC-NDR), has been observed in a range of transition metal oxides. Threshold switching devices exhibit a large non-linear change in electrical conductivity, switching from an insulating to a metallic state under external stimuli. Compact, scalable and low power threshold switching devices are of significant interest for use in existing and emerging technologies, including as a selector element in high-density memory arrays and as solid-state oscillators for hardware-based neuromorphic computing. This thesis explores the threshold switching in amorphous NbOx and the properties of individual and coupled oscillators based on this response. The study begins with an investigation of threshold switching in Pt/NbOx/TiN devices as a function device area, NbOx film thickness and temperature, which provides important insight into the structure of the self-assembled switching region. The devices exhibit combined threshold-memory behaviour after an initial voltage-controlled forming process, but exhibit symmetric threshold switching when the RESET and SET currents are kept below a critical value. In this mode, the threshold and hold voltages are shown to be independent of the device area and film thickness, and the threshold power, while independent of device area, is shown to decrease with increasing film thickness. These results are shown to be consistent with a structure in which the threshold switching volume is confined, both laterally and vertically, to the region between the residual memory filament and the electrode, and where the memory filament has a core-shell structure comprising a metallic core and a semiconducting shell. The veracity of this structure is demonstrated by comparing experimental results with the predictions of a resistor network model, and detailed finite element simulations. The next study focuses on electrical self-oscillation of an NbOx threshold switching device incorporated into a Pearson-Anson circuit configuration. Measurements confirm stable operation of the oscillator at source voltages as low as 1.06 V, and demonstrate frequency control in the range from 2.5 to 20.5 MHz with maximum frequency tuning range of 18 MHz/V. The oscillator exhibit three distinct oscillation regimes: sporadic spiking, stable oscillation and damped oscillation. The oscillation frequency, peak-to-peak amplitude and frequency are shown to be temperature and voltage dependent with stable oscillation achieved for temperatures up to ∼380 K. A physics-based threshold switching model with inclusion of device and circuit parameters is shown to explain the oscillation waveform and characteristic. The final study explores the oscillation dynamics of capacitively coupled Nb/Nb2O5 relaxation oscillators. The coupled system exhibits rich collective behaviour, from weak coupling to synchronisation, depending on the negative differential resistance response of the individual devices, the operating voltage and the coupling capacitance. These coupled oscillators are shown to exhibit stable frequency and phase locking states at source voltages as low as 2.2 V with MHz frequency tunable range. The numerical simulation of the coupled system highlights the role of source voltage, and circuit and device capacitance in controlling the coupling modes and dynamics