Meta-Heuristic Optimization Methods for Quaternion-Valued Neural Networks

In recent years, real-valued neural networks have demonstrated promising, and often striking, results across a broad range of domains. This has driven a surge of applications utilizing high-dimensional datasets. While many techniques exist to alleviate issues of high-dimensionality, they all induce a cost in terms of network size or computational runtime. This work examines the use of quaternions, a form of hypercomplex numbers, in neural networks. The constructed networks demonstrate the ability of quaternions to encode high-dimensional data in an efficient neural network structure, showing that hypercomplex neural networks reduce the number of total trainable parameters compared to their real-valued equivalents. Finally, this work introduces a novel training algorithm using a meta-heuristic approach that bypasses the need for analytic quaternion loss or activation functions. This algorithm allows for a broader range of activation functions over current quaternion networks and presents a proof-of-concept for future work

Meta-Heuristic Optimization Methods for Quaternion-Valued Neural Networks

In recent years, real-valued neural networks have demonstrated promising, and often striking, results across a broad range of domains. This has driven a surge of applications utilizing high-dimensional datasets. While many techniques exist to alleviate issues of high-dimensionality, they all induce a cost in terms of network size or computational runtime. This work examines the use of quaternions, a form of hypercomplex numbers, in neural networks. The constructed networks demonstrate the ability of quaternions to encode high-dimensional data in an efficient neural network structure, showing that hypercomplex neural networks reduce the number of total trainable parameters compared to their real-valued equivalents. Finally, this work introduces a novel training algorithm using a meta-heuristic approach that bypasses the need for analytic quaternion loss or activation functions. This algorithm allows for a broader range of activation functions over current quaternion networks and presents a proof-of-concept for future work

Neural Computing in Quaternion Algebra

兵庫県立大学201

Quaternion Neural Network with Temporal Feedback Calculation: Application to Cardiac Vector Velocity during Myocardial Infarction

Quaternion neural networks have been shownto be useful in image and signal processing applications.Herein, we propose a novel architecture of a neural unit model characterized by its ability of encoding 3-dimensional past information and that facilitates the learning of velocity patterns. We evaluate the implementation of the networkin a study of the cardiac vector velocity and its usefulness in early detection of patients with anterior myocardial infarction. Experimental results show an improvement of the performance in terms of convergence speed and precision when comparing with traditional methods. Furthermore, the network shows successful results in measuring the velocity reduction that is usually observed in vectorcardiogram signals in the presence of myocardial damage. Through a linear discriminant analysis, a pair of 100% / 98% of sensitivity/specificity is met with only two velocity parameters. We conclude that this method is a very promising developmentfor future computational tools devoted to early diagnosis ofheart diseases.Fil: Cruces, Pablo Daniel. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Saavedra 15. Instituto Argentino de Matemática Alberto Calderón; ArgentinaFil: Correa Prado, Raul Oscar. Universidad Nacional de San Juan. Facultad de Ingeniería. Departamento de Electrónica y Automática. Gabinete de Tecnología Médica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Laciar Leber, Eric. Universidad Nacional de San Juan. Facultad de Ingeniería. Departamento de Electrónica y Automática. Gabinete de Tecnología Médica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Arini, Pedro David. Universidad de Buenos Aires. Facultad de Ingeniería. Instituto de Ingeniería Biomédica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Saavedra 15. Instituto Argentino de Matemática Alberto Calderón; ArgentinaXXI Congreso Argentino de BioingenieríaCórdobaArgentinaSociedad Argentina de BioingenieríaUniversidad Nacional de Córdob

Enabling quaternion derivatives: the generalized HR calculus

Quaternion derivatives exist only for a very restricted class of analytic (regular) functions; however, in many applications, functions of interest are real-valued and hence not analytic, a typical case being the standard real mean square error objective function. The recent HR calculus is a step forward and provides a way to calculate derivatives and gradients of both analytic and non-analytic functions of quaternion variables; however, the HR calculus can become cumbersome in complex optimization problems due to the lack of rigorous product and chain rules, a consequence of the non-commutativity of quaternion algebra. To address this issue, we introduce the generalized HR (GHR) derivatives which employ quaternion rotations in a general orthogonal system and provide the left- and right-hand versions of the quaternion derivative of general functions. The GHR calculus also solves the long-standing problems of product and chain rules, mean-value theorem and Taylor's theorem in the quaternion field. At the core of the proposed GHR calculus is quaternion rotation, which makes it possible to extend the principle to other functional calculi in non-commutative settings. Examples in statistical learning theory and adaptive signal processing support the analysis

Applied Mathematics and Computational Physics

As faster and more efficient numerical algorithms become available, the understanding of the physics and the mathematical foundation behind these new methods will play an increasingly important role. This Special Issue provides a platform for researchers from both academia and industry to present their novel computational methods that have engineering and physics applications

Development of customized conversational interfaces with Deep Learning techniques

This Bachelor’s thesis will cover the end-to-end process of developing a personalized conversational interface for a specific domain, using Deep Learning techniques. In particular, it will focus on the study of the Dialog Manager module, which is in charge of deciding the next system response based on the current dialog state. AlthoughthereisplentyofliteratureaboutMachineLearningappliedtotheconstruction of dialog management models, there is very little reference to the utilization of Deep Learning for such task. As a result, this work analyzes the improvement that deep neural networks can bring to accuracy. Several models are created with TensorFlow, and comparisons are made with traditional Machine Learning solutions. Results show that Deep Learning is not the most recommended approach for this type of problems, yet further research is suggested for more complex datasets. After this, one of the Deep Learning models, based on a train scheduling domain, is used for the implementation of the dialog manager inside a real spoken dialog system. To integrate the rest of required components of such technology (automatic speech recognizer, natural language understanding module and text-to-speech service), a modern framework is used: DialogFlow. With this platform, a complete chatbot is built in the form of an assistant in the train scheduling domain. Evaluationof thespoken dialogsystemwith real users generatesavery positivefeedback, demonstrating that a Deep Learning based dialog manager is a valid solution in commercial conversational interfaces.Ingeniería Informátic

Quaternionic Multilayer Perceptron with Local Analyticity

A multi-layered perceptron type neural network is presented and analyzed in this paper. All neuronal parameters such as input, output, action potential and connection weight are encoded by quaternions, which are a class of hypercomplex number system. Local analytic condition is imposed on the activation function in updating neurons’ states in order to construct learning algorithm for this network. An error back-propagation algorithm is introduced for modifying the connection weights of the network