Meta-Heuristic Optimization Methods for Quaternion-Valued Neural Networks

In recent years, real-valued neural networks have demonstrated promising, and often striking, results across a broad range of domains. This has driven a surge of applications utilizing high-dimensional datasets. While many techniques exist to alleviate issues of high-dimensionality, they all induce a cost in terms of network size or computational runtime. This work examines the use of quaternions, a form of hypercomplex numbers, in neural networks. The constructed networks demonstrate the ability of quaternions to encode high-dimensional data in an efficient neural network structure, showing that hypercomplex neural networks reduce the number of total trainable parameters compared to their real-valued equivalents. Finally, this work introduces a novel training algorithm using a meta-heuristic approach that bypasses the need for analytic quaternion loss or activation functions. This algorithm allows for a broader range of activation functions over current quaternion networks and presents a proof-of-concept for future work

Optimization-Based Control Methodologies with Applications to Autonomous Vehicle

This thesis includes two main parts. In the first part, the main contribution is to develop nonsingular rigid-body attitude control laws using a convex formulation, and implement them in an experimental set up. The attitude recovery problem is first parameterized in terms of quaternions, and then two polynomial controllers using an SoS Lyapunov function and an SoS density function are developed. A quaternion-based polynomial controller using backstepping is also designed to make the closed-loop system asymptotically stable. Moreover, the proposed quaternion-based controllers are implemented in a Quanser helicopter, and compared to the polynomial controllers and a PID controller experimentally. The main contribution of the second part of this thesis is to analytically solve the Hamilton-Jacobi-Bellman equation for a class of third order nonlinear optimal control problems for which the dynamics are affine and the cost is quadratic in the input. One special advantage of this work is that the solution is directly obtained for the control input without the computation of a value function first. The value function can however also be obtained based on the control input. Furthermore, a Lyapunov function can be constructed for a subclass of optimal control problems, yielding a proof certificate for stability. Using the proposed methodology, experimental results of a path following problem implemented in a Wheeled Mobile Robot (WMR) are then presented to verify the effectiveness of the proposed methodology

Enabling quaternion derivatives: the generalized HR calculus

Quaternion derivatives exist only for a very restricted class of analytic (regular) functions; however, in many applications, functions of interest are real-valued and hence not analytic, a typical case being the standard real mean square error objective function. The recent HR calculus is a step forward and provides a way to calculate derivatives and gradients of both analytic and non-analytic functions of quaternion variables; however, the HR calculus can become cumbersome in complex optimization problems due to the lack of rigorous product and chain rules, a consequence of the non-commutativity of quaternion algebra. To address this issue, we introduce the generalized HR (GHR) derivatives which employ quaternion rotations in a general orthogonal system and provide the left- and right-hand versions of the quaternion derivative of general functions. The GHR calculus also solves the long-standing problems of product and chain rules, mean-value theorem and Taylor's theorem in the quaternion field. At the core of the proposed GHR calculus is quaternion rotation, which makes it possible to extend the principle to other functional calculi in non-commutative settings. Examples in statistical learning theory and adaptive signal processing support the analysis

Applied Mathematics and Computational Physics

As faster and more efficient numerical algorithms become available, the understanding of the physics and the mathematical foundation behind these new methods will play an increasingly important role. This Special Issue provides a platform for researchers from both academia and industry to present their novel computational methods that have engineering and physics applications