Identification and Optimal Linear Tracking Control of ODU Autonomous Surface Vehicle

Autonomous surface vehicles (ASVs) are being used for diverse applications of civilian and military importance such as: military reconnaissance, sea patrol, bathymetry, environmental monitoring, and oceanographic research. Currently, these unmanned tasks can accurately be accomplished by ASVs due to recent advancements in computing, sensing, and actuating systems. For this reason, researchers around the world have been taking interest in ASVs for the last decade. Due to the ever-changing surface of water and stochastic disturbances such as wind and tidal currents that greatly affect the path-following ability of ASVs, identification of an accurate model of inherently nonlinear and stochastic ASV system and then designing a viable control using that model for its planar motion is a challenging task. For planar motion control of ASV, the work done by researchers is mainly based on the theoretical modeling in which the nonlinear hydrodynamic terms are determined, while some work suggested the nonlinear control techniques and adhered to simulation results. Also, the majority of work is related to the mono- or twin-hull ASVs with a single rudder. The ODU-ASV used in present research is a twin-hull design having two DC trolling motors for path-following motion. A novel approach of time-domain open-loop observer Kalman filter identifications (OKID) and state-feedback optimal linear tracking control of ODU-ASV is presented, in which a linear state-space model of ODU-ASV is obtained from the measured input and output data. The accuracy of the identified model for ODU-ASV is confirmed by validation results of model output data reconstruction and benchmark residual analysis. Then, the OKID-identified model of the ODU-ASV is utilized to design the proposed controller for its planar motion such that a predefined cost function is minimized using state and control weighting matrices, which are determined by a multi-objective optimization genetic algorithm technique. The validation results of proposed controller using step inputs as well as sinusoidal and arc-like trajectories are presented to confirm the controller performance. Moreover, real-time water-trials were performed and their results confirm the validity of proposed controller in path-following motion of ODU-ASV

Finding Critical and Gradient-Flat Points of Deep Neural Network Loss Functions

Despite the fact that the loss functions of deep neural networks are highly non-convex, gradient-based optimization algorithms converge to approximately the same performance from many random initial points. This makes neural networks easy to train, which, combined with their high representational capacity and implicit and explicit regularization strategies, leads to machine-learned algorithms of high quality with reasonable computational cost in a wide variety of domains. One thread of work has focused on explaining this phenomenon by numerically characterizing the local curvature at critical points of the loss function, where gradients are zero. Such studies have reported that the loss functions used to train neural networks have no local minima that are much worse than global minima, backed up by arguments from random matrix theory. More recent theoretical work, however, has suggested that bad local minima do exist. In this dissertation, we show that one cause of this gap is that the methods used to numerically find critical points of neural network losses suffer, ironically, from a bad local minimum problem of their own. This problem is caused by gradient-flat points, where the gradient vector is in the kernel of the Hessian matrix of second partial derivatives. At these points, the loss function becomes, to second order, linear in the direction of the gradient, which violates the assumptions necessary to guarantee convergence for second order critical point-finding methods. We present evidence that approximately gradient-flat points are a common feature of several prototypical neural network loss functions

The Shallow and the Deep:A biased introduction to neural networks and old school machine learning

The Shallow and the Deep is a collection of lecture notes that offers an accessible introduction to neural networks and machine learning in general. However, it was clear from the beginning that these notes would not be able to cover this rapidly changing and growing field in its entirety. The focus lies on classical machine learning techniques, with a bias towards classification and regression. Other learning paradigms and many recent developments in, for instance, Deep Learning are not addressed or only briefly touched upon.Biehl argues that having a solid knowledge of the foundations of the field is essential, especially for anyone who wants to explore the world of machine learning with an ambition that goes beyond the application of some software package to some data set. Therefore, The Shallow and the Deep places emphasis on fundamental concepts and theoretical background. This also involves delving into the history and pre-history of neural networks, where the foundations for most of the recent developments were laid. These notes aim to demystify machine learning and neural networks without losing the appreciation for their impressive power and versatility