29 research outputs found
Neural Network iLQR: A New Reinforcement Learning Architecture
As a notable machine learning paradigm, the research efforts in the context
of reinforcement learning have certainly progressed leaps and bounds. When
compared with reinforcement learning methods with the given system model, the
methodology of the reinforcement learning architecture based on the unknown
model generally exhibits significantly broader universality and applicability.
In this work, a new reinforcement learning architecture is developed and
presented without the requirement of any prior knowledge of the system model,
which is termed as an approach of a "neural network iterative linear quadratic
regulator (NNiLQR)". Depending solely on measurement data, this method yields a
completely new non-parametric routine for the establishment of the optimal
policy (without the necessity of system modeling) through iterative refinements
of the neural network system. Rather importantly, this approach significantly
outperforms the classical iterative linear quadratic regulator (iLQR) method in
terms of the given objective function because of the innovative utilization of
further exploration in the methodology. As clearly indicated from the results
attained in two illustrative examples, these significant merits of the NNiLQR
method are demonstrated rather evidently.Comment: 13 pages, 9 figure
Real-time collision avoidance for autonomous air vehicles
Thesis (M.S.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 1998.Includes bibliographical references (p. 138-139).by Christopher P. Sanders.M.S
Design and analysis of robust controllers for directional drilling tools
Directional drilling is a very important tool for the development of oil and gas deposits.
Attitude control which enables directional drilling for the efficient placement of the directional drilling tools in petroleum producing zones is reviewed along with the various
engineering requirements or constraints. This thesis explores a multivariable attitude governing plant model as formulated in Panchal et al. (2010) which is used for developing
robust control techniques. An inherent input and measurement delay which accounts for
the plant's dead-time is included in the design of the controllers. A Smith Predictor controller is developed for reducing the effect of this dead-time. The developed controllers
are compared for performance and robustness using structured singular value analysis and
also for their performance indicated by the transient response of the closed loop models. Results for the transient non-linear simulation of the proposed controllers are also
presented. The results obtained indicate that the objectives are satisfactorily achieved
Control Methods for High-Speed Supercavitating Vehicles
Supercavitation is an emerging technology that enables underwater vehicles to reach un- precedented speed. With proper design of cavitator attached to the vehicle nose, the vehicle body is surrounded by water vapor cavity, eliminating skin friction drag. This technology offers unprecedented drag reduction, though poses problems for vehicle design. The gas bubble surrounding the hull introduces highly coupled dynamic behavior, representing a challenge for the control designer. Development of stable, controllable supercavitating vehi- cles requires solution for several open problems. This dissertation addresses the problem of control oriented modeling, stability augmentation, and reference tracking using parameter dependent control techniques for supercavitating vehicles.\ud
The thesis is divided into three parts. A nonlinear dynamical model capturing the most important properties of the vehicle motion is developed from a control design perspective. The model includes memory effects associated with the time evolution of the cavity and uses lookup tables to determine forces.\ud
To aid understanding the cavity-vehicle interaction, a longitudinal control scenario is developed for a simplified longitudinal dynamical model with guaranteed properties. Sig- nificant insight is gained on planing behavior and operating envelope using constrained control inputs.\ud
Extending the longitudinal control problem, a linear parameter varying model of the coupled motion is developed to provide a platform for parameter dependent control syn- thesis. The mathematical model is scheduled with aerodynamic angles, uses steady-state approximation of the cavity, leading to uncertainty in the governing equations. Two Linear Parameter Varying (LPV) controllers are synthesized for the angle rate tracking problem, taking uncertainty into account. One uses traditional decoupled loops for pitch-, roll- and yaw-rate tracking. Ignoring the cross coupling, leads to more tractable subproblems . A controller, taking advantage of the coupling, is also presented in the thesis. The complexity of the coupled dynamics prohibits the synthesis of the controller as a single entity. Sev- eral LPV controllers synthesized for smaller overlapping regions of the parameter space are blended together, providing a single controller for the full flight envelope. Time-domain simulations of different vehicle-controller configurations, implemented on high-fidelity sim- ulations, provide insight into the capabilities of the supercavitating vehicle
Analytic Construction of Periodic Orbits in the Restricted Three-Body Problem
This dissertation explores the analytical solution properties surrounding a nominal periodic orbit in two different planes, the plane of motion of the two primaries and a plane perpendicular to the line joining the two primaries, in the circular restricted three-body problem. Assuming motion can be maintained in the plane and motion of the third body is circular, Jacobi\u27s integral equation can be analytically integrated, yielding a closed-form expression for the period and path expressed with elliptic integral and elliptic function theory. In this case, the third body traverses a circular path with nonuniform speed. In a strict sense, the in-plane assumption cannot be maintained naturally. However, there may be cases where the assumption is approximately maintained over a finite time period. More importantly, the nominal solution can be used as the basis for an iterative analytical solution procedure for the three dimensional periodic trajectory where corrections are computable in closed-form. In addition, the in-plane assumption can be strictly enforced with the application of modulated thrust acceleration. In this case, the required thrust control inputs are found to be nonlinear functions in time. Total velocity increment, required to maintain the nominal orbit, for one complete period of motion of the third body is expressed as a function of the orbit characteristics
A motion planning approach to protein folding
Protein folding is considered to be one of the grand challenge problems in biology. Protein folding refers to how a protein's amino acid sequence, under certain physiological conditions, folds into a stable close-packed three-dimensional structure known as the native state. There are two major problems in protein folding. One, usually called protein structure prediction, is to predict the structure of the protein's native state given only the amino acid sequence. Another important and strongly related problem, often called protein folding, is to study how the amino acid sequence dynamically transitions from an unstructured state to the native state. In this dissertation, we concentrate on the second problem. There are several approaches that have been applied to the protein folding problem, including molecular dynamics, Monte Carlo methods, statistical mechanical models, and lattice models. However, most of these approaches suffer from either overly-detailed simulations, requiring impractical computation times, or overly-simplified models, resulting in unrealistic solutions.
In this work, we present a novel motion planning based framework for studying protein folding. We describe how it can be used to approximately map a protein's energy landscape, and then discuss how to find approximate folding pathways and kinetics on this approximate energy landscape. In particular, our technique can produce potential energy landscapes, free energy landscapes, and many folding pathways all from a single roadmap. The roadmap can be computed in a few hours on a desktop PC using a coarse potential energy function. In addition, our motion planning based approach is the first simulation method that enables the study of protein folding kinetics at a level of detail that is appropriate (i.e., not too detailed or too coarse) for capturing possible 2-state and 3-state folding kinetics that may coexist in one protein. Indeed, the unique ability of our method to produce large sets of unrelated folding pathways may potentially provide crucial insight into some aspects of folding kinetics that are not available to other theoretical techniques
Mecánica Discreta para Sistemas Forzados y Ligados
Tesis inédita de la Universidad Complutense de Madrid, Facultad de Ciencias Matemáticas, leída el 10/07/2019Geometric mechanics is a branch of mathematics that studies classical mechanics of particles and fields from the point of view of geometry and its relation to symmetry. One of its most interesting developments was bringing together numerical analysis and geometry by relating what is known as discrete mechanics with numerical integration. This is called geometric integration. In the last 30 years this latter field has exploded with researchfrom the purely theoretical to the strictly applied. Variational integrators are a type of geometric integrators arising naturally from the discretization process of variational principles in mechanics. They display some of the most salient features of the theory, such as symplecticity, preservation of momenta and quasi-preservation of energy. These methods also apply very naturally to optimal control problems, also based on variational principles. Unfortunately, not all mechanical systems of interest admit a variational formulation. Such is the case of forced and nonholonomic mechanical systems. In this thesis we study both of these types of systems and obtain several new results. By geometrizing a new technique of duplication of variables and applying it, we were able to definitely prove the order of integrators for forced systems by using only variational techniques. Furthermore, we could also extend these results to the reduced setting in Lie groups, leading us to a very interesting geometric structure, Poisson groupoids. In addition, we developed new methods to geometrically integrate nonholonomic systems to arbitrary order preserving their constraints exactly. These methods can be seen as nonholonomic extensions of variational methods, and we were able to prove their order, although not through variational means. These methods have a nice geometric interpretation and thanks to their closeness to variational methods, they can be easily generalized to other geometric settings, such as Lie group integration. Finally, we were able to apply these new methods to optimal control problems...La mecánica clásica es un campo tan fundamental para la física como la geometría lo es para las matemáticas. Ambos están interrelacionados y su estudio conjunto así como sus interacciones forman lo que hoy se conoce como la mecánica geométrica [véase, por ejemplo, AM78; Arn89; Hol11a; Hol11 b]. Hoy es bien sabido que el concepto de simetría tiene importantes consecuencias para los sistemas mecánicos. En particular, la evolución de los sistemas mecánicos suele mostrar ciertas propiedades de preservación en forma de cantidades conservadas del movimiento o preservación de estructuras geométricas. Ser capaces de capturar estas propiedades es vital para tener una imagen fiel, tanto en términos cuantitativos como cualitativos, de cara al estudio de estos sistemas. Esto tiene gran importancia en el campo teórico y también el aplicado, como en la ingeniería. La experimentación en laboratorios y la generación de prototipos son procesos costosos y que requieren de tiempo, y para determinad os sistemas pueden no ser siquiera factibles. Con la llegada el ordenador, simular y experimentar con sistemas mecánicos de forma rápida y económica se convirtió en una realidad . Desde sencillas simulaciones balísticas para alumnos de secundaria a simulaciones de dinámica molecular a gran escala; desde la planificación de trayectorias para vehículos autónomos a la estimación de movimientos en robots bípedos; desde costosas simulaciones basadas en modelos físicos para la industria de la animación a la simulación de sólidos rígidos y deformables en tiempo real para la industria del videojuego, el tratamiento numérico de sistemas de complejidad creciente se ha convertido en una necesidad. Naturalmente surgieron nuevos algoritmos capaces de conservar gran parte de las propiedades geométricas de estos sistemas, configurando lo que a hora se conoce como integración geométrica [véase SC94; HLW1O]. En los últimos 20 a 30 años se han dado grandes pasos en esta dirección, con el desarrollo de métodos que conservan energía, métodos simplécticos y multisimplécticos, métodos que preservan el espacio de configuración y más. Aún así, la investigación en esta área está todavía lejos de acabar. Por ejemplo , los sistemas sometidos a fuerzas externas y con ligaduras ofrecen ciertas dificultades que han de ser abordadas, y esta tesis se dedica a explorar estos dos casos ofreciendo nuevos desarrollos y resultados...Fac. de Ciencias MatemáticasTRUEunpu