On two-sided controls of a linear diffusion

siirretty Doriast

Markov Decision Processes

The theory of Markov Decision Processes is the theory of controlled Markov chains. Its origins can be traced back to R. Bellman and L. Shapley in the 1950\u27s. During the decades of the last century this theory has grown dramatically. It has found applications in various areas like e.g. computer science, engineering, operations research, biology and economics. In this article we give a short introduction to parts of this theory. We treat Markov Decision Processes with finite and infinite time horizon where we will restrict the presentation to the so-called (generalized) negative case. Solution algorithms like Howard\u27s policy improvement and linear programming are also explained. Various examples show the application of the theory. We treat stochastic linear-quadratic control problems, bandit problems and dividend pay-out problems

Optimal Stopping of Gauss-Markov processes

Mención Internacional en el título de doctorIn this thesis we contribute to the optimal stopping theory literature, in the time-inhomogeneous framework, by solving Optimal Stopping Problems (OSPs) related to Gauss–Markov (GM) processes, both when they are non-degenerated, and when they are pinned to a deterministic value at a terminal time. For pinned processes, we bypassed the challenge of their explosive drifts by equating them to a time-space-transformed Brownian Motion (BM). For each OSP, we characterized the free-boundary equation as the unique solution of a type-two Volterra integral equation. The value functions were, then, expressed as an integral of the OSBs. We used a solution methodology in the spirit of Peskir (2005). That is, a direct, probabilistic approach that harvests sufficient smoothness of the value function and the Optimal Stopping Boundary (OSB) to solve the associated free-boundary problem by using an extended Itô’s lemma. In doing so, we proved the Lipschitz continuity of the OSB away from the horizon. This result extends the technique in De Angelis and Stabile (2019) and blueprints a methodology to obtain similar smoothness on other OSPs. Another highly customizable technique was the one we employed to obtain the OSB’s boundedness. By comparing the non-degenerated GM process and the Gauss Markov Bridge (GMB) with a BM and a Brownian Bridge (BB), respectively, we found bounds for the OSBs of the former two processes from those of the latter two. Two different fixed-point algorithms were presented and implemented to solve the freeboundary equation. One based on backward induction (see Section 3.4) and one based on the Picard iteration method (see Sections 2.5, 4.6, and 5.6). With the aid of these algorithms, we illustrated the geometry of the OSB for different forms of the processes’ drift and volatility (see Figures 2.1, 3.1, 4.1–4.3, and 5.2). It is worth mentioning the statistical inference study we perform on the OSB in the BB case (see Section 3.4), as this is not a typical subject addressed in optimal stopping theory, and it is potentially extensible to tackle more general settings where likelihood theory is worked out. Indeed, the methodology consists in using the asymptotic normality of the BB volatility’s maximum-likelihood estimate to extend, by using the delta method, such property to the OSB plugin estimator. This allowed us to provide (point-wise) confidence curves for the OSB. We also offer a financial perspective of our work in Chapters 2 and 3, by linking the OSPs to the problem of optimally exercising American options. Remarkably, in Section 3.5, we show the competitiveness of the BB model against the geometric BM in this regard, when the option is written on IBM’s and Apple’s stocks, and in the presence of the pinning-at-the-strike effect. In addition, the confidence curves computed in Section 3.4 provide traders with a mechanism to introduce a risk-preference element.Programa de Doctorado en Ingeniería Matemática por la Universidad Carlos III de MadridPresidente: Franciso de Asís Torres Ruiz.- Secretaria: Rosa Elvira Lillo Rodríguez.- Vocal: Tiziano De Angeli

Performance optimization for unmanned vehicle systems

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2008.Includes bibliographical references (p. 149-157).Technological advances in the area of unmanned vehicles are opening new possibilities for creating teams of vehicles performing complex missions with some degree of autonomy. Perhaps the most spectacular example of these advances concerns the increasing deployment of unmanned aerial vehicles (UAVs) in military operations. Unmanned Vehicle Systems (UVS) are mainly used in Information, Surveillance and Reconnaissance missions (ISR). In this context, the vehicles typically move about a low-threat environment which is sufficiently simple to be modeled successfully. This thesis develops tools for optimizing the performance of UVS performing ISR missions, assuming such a model.First, in a static environment, the UVS operator typically requires that a vehicle visit a set of waypoints once or repetitively, with no a priori specified order. Minimizing the length of the tour traveled by the vehicle through these waypoints requires solving a Traveling Salesman Problem (TSP). We study the TSP for the Dubins' vehicle, which models the limited turning radius of fixed wing UAVs. In contrast to previously proposed approaches, our algorithms determine an ordering of the waypoints that depends on the model of the vehicle dynamics. We evaluate the performance gains obtained by incorporating such a model in the mission planner.With a dynamic model of the environment the decision making level of the UVS also needs to solve a sensor scheduling problem. We consider M UAVs monitoring N > M sites with independent Markovian dynamics, and treat two important examples arising in this and other contexts, such as wireless channel or radar waveform selection. In the first example, the sensors must detect events arising at sites modeled as two-state Markov chains. In the second example, the sites are assumed to be Gaussian linear time invariant (LTI) systems and the sensors must keep the best possible estimate of the state of each site.(cont.) We first present a bound on the achievable performance which can be computed efficiently by a convex program, involving linear matrix inequalities in the LTI case. We give closed-form formulas for a feedback index policy proposed by Whittle. Comparing the performance of this policy to the bound, it is seen to perform very well in simulations. For the LTI example, we propose new open-loop periodic switching policies whose performance matches the bound.Ultimately, we need to solve the task scheduling and motion planning problems simultaneously. We first extend the approach developed for the sensor scheduling problems to the case where switching penalties model the path planning component. Finally, we propose a new modeling approach, based on fluid models for stochastic networks, to obtain insight into more complex spatiotemporal resource allocation problems. In particular, we give a necessary and sufficient stabilizability condition for the fluid approximation of the problem of harvesting data from a set of spatially distributed queues with spatially varying transmission rates using a mobile server.by Jerome Le Ny.Ph.D

Dynamical Systems

Complex systems are pervasive in many areas of science integrated in our daily lives. Examples include financial markets, highway transportation networks, telecommunication networks, world and country economies, social networks, immunological systems, living organisms, computational systems and electrical and mechanical structures. Complex systems are often composed of a large number of interconnected and interacting entities, exhibiting much richer global scale dynamics than the properties and behavior of individual entities. Complex systems are studied in many areas of natural sciences, social sciences, engineering and mathematical sciences. This special issue therefore intends to contribute towards the dissemination of the multifaceted concepts in accepted use by the scientific community. We hope readers enjoy this pertinent selection of papers which represents relevant examples of the state of the art in present day research. [...

Mathematical and Numerical Aspects of Dynamical System Analysis

From Preface: This is the fourteenth time when the conference “Dynamical Systems: Theory and Applications” gathers a numerous group of outstanding scientists and engineers, who deal with widely understood problems of theoretical and applied dynamics. Organization of the conference would not have been possible without a great effort of the staff of the Department of Automation, Biomechanics and Mechatronics. The patronage over the conference has been taken by the Committee of Mechanics of the Polish Academy of Sciences and Ministry of Science and Higher Education of Poland. It is a great pleasure that our invitation has been accepted by recording in the history of our conference number of people, including good colleagues and friends as well as a large group of researchers and scientists, who decided to participate in the conference for the first time. With proud and satisfaction we welcomed over 180 persons from 31 countries all over the world. They decided to share the results of their research and many years experiences in a discipline of dynamical systems by submitting many very interesting papers. This year, the DSTA Conference Proceedings were split into three volumes entitled “Dynamical Systems” with respective subtitles: Vibration, Control and Stability of Dynamical Systems; Mathematical and Numerical Aspects of Dynamical System Analysis and Engineering Dynamics and Life Sciences. Additionally, there will be also published two volumes of Springer Proceedings in Mathematics and Statistics entitled “Dynamical Systems in Theoretical Perspective” and “Dynamical Systems in Applications”

Elliptic partial differential equations from an elementary viewpoint

These notes are the outcome of some courses taught to undergraduate and graduate students from the University of Western Australia, the Pontif\'{\i}cia Universidade Cat\'olica do Rio de Janeiro, the Indian Institute of Technology Gandhinagar and the Ukrainian Catholic University in 2021 and 2022

Flight Mechanics/Estimation Theory Symposium

Onboard and real time image processing to enhance geometric correction of the data is discussed with application to autonomous navigation and attitude and orbit determination. Specific topics covered include: (1) LANDSAT landmark data; (2) star sensing and pattern recognition; (3) filtering algorithms for Global Positioning System; and (4) determining orbital elements for geostationary satellites