Optimal trajectory management for aircraft descent operations subject to time constraints

The growth in traffic increased the pressure on the environmental sustainability of air transport. In this context, many research effort has been devoted to minimise the environmental impact in the different phases of flight. Continuous descent operations, ideally performed with the engines at idle from the cruise altitude to right before landing, have shown to reduce fuel, noise nuisance and gaseous emissions if compared to conventional descents. However, this type of operations suffer from a well known drawback: the loss of predictability from the air traffic control point of view in terms of overfly times at the different waypoints of the route. Due to this loss of predictability, air traffic controllers require large separation buffers, thus reducing the capacity of the airport. Previous works investigating this issue showed that the ability to meet a controlled time of arrival at a metering fix could enable continuous descent operations while simultaneously maintaining airport throughput. In this context, the planning and guidance functions of state-of-the-art flight management systems need to be modernised. On-board trajectory planners capable to generate an optimal trajectory plan satisfying time constraints introduced during the flight are seldom, mainly because the real-time optimisation of aircraft trajectories is still elusive. Furthermore, the time scale and spatial resolution of the wind forecasts used by these trajectory planners are far from being adequate to generate accurate flight time predictions. Finally, there exist guidance strategies capable to accurately comply with time constraints enforced at a certain fix in the trajectory plan, yet they are not specifically designed to minimise the environmental impact. This PhD thesis aims at investigating fast optimisation techniques to enable real-time updates of the optimal trajectory plan subject to time constraints during the descent; wind networking concepts to generate more accurate and up-to-date wind forecasts and, consequently, time predictions; and more robust an efficient guidance strategies to reduce the environmental impact at the maximum extent while complying with the time constraints of the trajectory plan. First, the feasible time window at a metering fix that could be achieved during a descent requiring neither thrust nor speed brakes usage is quantified as a function of the aircraft states (altitude, distance to the metering fix and airspeed), aiming to assess the feasibility of guidance strategies that take advantage of time and energy management concepts. Then, the performance of four of these guidance strategies is compared in terms of environmental impact mitigation and ability to satisfy operational constraints. Results from the comparison reveal that model predictive control, a strategy based on a frequent re-calculation of the optimal trajectory plan during the execution of the descent, is the most robust in terms of energy and time deviation at the metering fix, providing at the same time excellent environmental impact mitigation figures. However, the execution time required to solve a rigorous trajectory optimisation problem at each re-calculation instant remains a critical limitation for practical applications. In order to address this issue, a variant of the model predictive control strategy that allows for fast updates of the optimal trajectory plan based on parametric sensitivities is proposed, which shows analogous results yet halving the time needed to update the optimal trajectory plan. Finally, the potential benefits of using wind observations broadcast by nearby aircraft to reconstruct the wind profile downstream right before updating the optimal trajectory plan when using model predictive control is also investigated. Promising results show that the combination of model predictive control with wind networking concepts could enable optimal descents without degrading the capacity of the airport.El creixement del trànsit ha augmentat la pressió sobre la sostenibilitat ambiental del transport aeri. En aquest àmbit s'han dedicat molts esforços en recerca per reduir l'impacte ambiental en les diferents fases del vol. Les operacions de descens continu, en les quals l'aeronau descendeix amb els motors a ralentí des de l'altitud de creuer fins just abans d'aterrar, han demostrat ser una solució atractiva per reduir el combustible, el soroll i les emissions en la fase de descens. Desafortunadament, aquest tipus d'operacions tenen un inconvenient molt important: la pèrdua de predictibilitat des del punt de vista dels controladors de trànsit aeri, en termes de temps de sobrevol als diferents punts de pas de la ruta. Per aquesta raó, els controladors necessiten aplicar més separació entre aeronaus, reduint així la capacitat de l'aeroport. Treballs anteriors han demostrat que si les aeronaus fossin capaces de satisfer restriccions de temps de sobrevol a un o més punts de pas, seria possible implementar operacions de descens continu sense degradar la capacitat de l'aeroport. Malauradament, avui en dia existeixen pocs sistemes de gestió de vol capaços de generar trajectòries òptimes que satisfacin restriccions de temps, principalment perquè l’optimització de trajectòries en temps real continua sent una tasca difícil. A més, la resolució espacial i temporal dels models de vent utilitzades per els planificadors de trajectòria no son suficients per generar prediccions de temps de sobrevol prou fiables. Finalment, les estratègies de guiatge que fins i tot avui en dia permetrien satisfer amb exactitud restriccions de temps de sobrevol, no estan dissenyades específicament per minimitzar l’impacte ambiental. Aquesta tesi té com a objectiu explorar algoritmes de d'optimització ràpids i robustos que permetin actualitzar la trajectòria òptima en temps real durant l'execució del descens, satisfent al mateix temps restriccions de temps de sobrevol; també s'investigaran nous conceptes de que permetin generar models de vent molt exactes a partir d'observacions emeses per aeronaus veïnes; i estratègies de guiatge més intel·ligents que minimitzin l'impacte ambiental de les operacions de descens continu subjectes a restriccions de temps de sobrevol. En primer lloc, es quantifica la finestra de temps disponible al punt on s'aplica la restricció de temps de sobrevol, en funció dels estats de l'aeronau (altitud, velocitat i distància al punt) i assumint que els motors es mantenen ralentí i que no s'utilitzen aerofrens durant tot el descens. Els resultats de l'experiment indiquen que es podrien utilitzar estratègies de guiatge que gestionessin l'energia cinètica i potencial de l'aeronau per satisfer restriccions de temps sense necessitat de gastar més combustible. A continuació, es compararen quatre d'aquestes estratègies. Els resultats d'aquests segon experiment indiquen que el control predictiu, una estratègia que contínuament actualitza la trajectòria òptima durant el descens, es molt robusta en termes d'errors de temps i energia, i que també redueix l'impacte ambiental. Malauradament, es tarda massa a actualitzar la trajectòria òptima cada cop que s’actualitza, fet que limita la implementació d'aquesta estratègia. Per tal d'afrontar aquesta limitació, es proposa una variant que utilitza sensitivitats paramètriques per reduir el temps d'execució a l'hora d'actualitzar la trajectòria òptima, sense degradar significativament la seva exactitud. Finalment, s'investiguen els possibles beneficis d'aprofitar observacions de vent enviades per les aeronaus del volant per millorar el model de vent i, conseqüentment, l'exactitud de la trajectòria calculada. Resultats prometedors demostren que si s’implementés model predictiu com a estratègia de guiatge i les aeronaus cooperessin per compartir observacions de vent, es reduiria l'impacte ambiental sense degradar la capacitat de l'aeroport.Postprint (published version

Adaptive Order Dispatching based on Reinforcement Learning: Application in a Complex Job Shop in the Semiconductor Industry

Heutige Produktionssysteme tendieren durch die Marktanforderungen getrieben zu immer kleineren Losgrößen, höherer Produktvielfalt und größerer Komplexität der Materialflusssysteme. Diese Entwicklungen stellen bestehende Produktionssteuerungsmethoden in Frage. Im Zuge der Digitalisierung bieten datenbasierte Algorithmen des maschinellen Lernens einen alternativen Ansatz zur Optimierung von Produktionsabläufen. Aktuelle Forschungsergebnisse zeigen eine hohe Leistungsfähigkeit von Verfahren des Reinforcement Learning (RL) in einem breiten Anwendungsspektrum. Im Bereich der Produktionssteuerung haben sich jedoch bisher nur wenige Autoren damit befasst. Eine umfassende Untersuchung verschiedener RL-Ansätze sowie eine Anwendung in der Praxis wurden noch nicht durchgeführt. Unter den Aufgaben der Produktionsplanung und -steuerung gewährleistet die Auftragssteuerung (order dispatching) eine hohe Leistungsfähigkeit und Flexibilität der Produktionsabläufe, um eine hohe Kapazitätsauslastung und kurze Durchlaufzeiten zu erreichen. Motiviert durch komplexe Werkstattfertigungssysteme, wie sie in der Halbleiterindustrie zu finden sind, schließt diese Arbeit die Forschungslücke und befasst sich mit der Anwendung von RL für eine adaptive Auftragssteuerung. Die Einbeziehung realer Systemdaten ermöglicht eine genauere Erfassung des Systemverhaltens als statische Heuristiken oder mathematische Optimierungsverfahren. Zusätzlich wird der manuelle Aufwand reduziert, indem auf die Inferenzfähigkeiten des RL zurückgegriffen wird. Die vorgestellte Methodik fokussiert die Modellierung und Implementierung von RL-Agenten als Dispatching-Entscheidungseinheit. Bekannte Herausforderungen der RL-Modellierung in Bezug auf Zustand, Aktion und Belohnungsfunktion werden untersucht. Die Modellierungsalternativen werden auf der Grundlage von zwei realen Produktionsszenarien eines Halbleiterherstellers analysiert. Die Ergebnisse zeigen, dass RL-Agenten adaptive Steuerungsstrategien erlernen können und bestehende regelbasierte Benchmarkheuristiken übertreffen. Die Erweiterung der Zustandsrepräsentation verbessert die Leistung deutlich, wenn ein Zusammenhang mit den Belohnungszielen besteht. Die Belohnung kann so gestaltet werden, dass sie die Optimierung mehrerer Zielgrößen ermöglicht. Schließlich erreichen spezifische RL-Agenten-Konfigurationen nicht nur eine hohe Leistung in einem Szenario, sondern weisen eine Robustheit bei sich ändernden Systemeigenschaften auf. Damit stellt die Forschungsarbeit einen wesentlichen Beitrag in Richtung selbstoptimierender und autonomer Produktionssysteme dar. Produktionsingenieure müssen das Potenzial datenbasierter, lernender Verfahren bewerten, um in Bezug auf Flexibilität wettbewerbsfähig zu bleiben und gleichzeitig den Aufwand für den Entwurf, den Betrieb und die Überwachung von Produktionssteuerungssystemen in einem vernünftigen Gleichgewicht zu halten

Modelling of a rope-free passenger transportation system for active cabin vibration damping

Conventional vertical passenger transportation is performed by lifts. Conventional traction-drive electrical lifts use ropes to transfer the rotational motion of an electrical motor into a vertical motion of the cabin. The vertical passenger transportation system discussed in this paper does not use any ropes, the motor directly provides a driving force, which moves the cabin. This new propulsion is realized through an electrical linear motor. The use of the linear motor requires a new design of the passenger transportation system (PTS), which includes reducing the weight of the car through lightweight construction. The reduced stiffness of the lightweight design renders the construction more vulnerable to vibrations. In order to improve ride quality of the transportation system it is necessary to develop new concepts to damp the vibrations. One way to increase stiffness characteristics of the system is to introduce active damping components to be used alongside passive damping components. It is essential to derive a dynamic model of the system in order to design and also later control these damping components in the best possible way. This paper describes the fundamental steps undertaken to derive a dynamic model for designing and controlling active damping components for the new type of vertical PTS. The model is derived as a Multi-Body System (MBS), where the connections between the bodies are modelled as spring damper elements. The derivation of the MBS is demonstrated on a transportation system, consisting of three main components: a sledge, holding the rotor of the linear motor; a mounting frame, which is used to provide support for the cabin; and the actual cabin. The modelling of the propulsion system, thus the electrical part of the PTS, will not be the focus of this work

Evaluating a holistic energy benchmarking parameter of lift systems by using computer simulation

At present, there are benchmarking parameters to assess the energy performance of lifts, e.g. one in Germany adopted by VDI (4707-1/2), one internationally published by ISO (BS EN ISO 25745-2:2015), and the other in Hong Kong adopted by The Hong Kong Special Administrative Region (HKSAR) Government. These parameters are mainly checking the energy consumed by a lift drive without considering real time passenger demands and traffic conditions; the one in Hong Kong pinpointing a fully loaded up-journey under rated speed and the two in Europe pinpointing a round trip, bottom floor to top floor and return with an empty car, though including energy consumed by lighting, displays, ventilation etc. A holistic normalization method by Lam et al [1] was developed a number of years ago by one of the co-authors of this article, which can assess both drive efficiency and traffic control, termed J/kg-m, which is now adopted by the HKSAR Government as a good practice, but not specified in the mandatory code. In Europe, the energy unit of Wh has been used but here, Joule (J), i.e. Ws, is adopted to discriminate the difference between the two concepts. In this article, this parameter is evaluated under different lift traffic scenarios using computer simulation techniques, with an aim of arriving at a reasonable figure for benchmarking an energy efficient lift system with both an efficient drive as well as an efficient supervisory traffic control

A study into the influence of the car geometry on the aerodynamic transient effects arising in a high rise lift installation

One of the main goals in designing a high-speed lift system is developing a more aerodynamically efficient car geometry that guarantees a good ride comfort and reduces the energy consumption. In this study, a three-dimensional computational fluid dynamics (CFD) model has been developed to analyse an unsteady turbulent air flow around two cars moving in a lift shaft. The paper is focused on transient aerodynamic effects arising when two cars pass each other in the same shaft at the same speed. The scenarios considered in the paper involve cars having three different geometries. Aerodynamic forces such as the drag force that occur due to the vertical opposite motions of the cars have been investigated. Attention is paid to the airflow velocity and pressure distribution around the car structures. The flow pattern in the boundary layer around each car has been calculated explicitly to examine the flow separation in the wake region. The results presented in the paper would be useful to guide the lift designers to understand and mitigate the aerodynamic effects arising in the lift shaft

Efficient model learning for dialog management

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 118-122).Partially Observable Markov Decision Processes (POMDPs) have succeeded in many planning domains because they can optimally trade between actions that will increase an agent's knowledge about its environment and actions that will increase an agent's reward. However, POMDPs are defined with a large number of parameters which are difficult to specify from domain knowledge, and gathering enough data to specify the parameters a priori may be expensive. This work develops several efficient algorithms for learning the POMDP parameters online and demonstrates them on dialog manager for a robotic wheelchair. In particular, we show how a combination of specialized queries ("meta-actions") can enable us to create a robust dialog manager that avoids the pitfalls in other POMDP-learning approaches. The dialog manager's ability to reason about its uncertainty -- and take advantage of low-risk opportunities to reduce that uncertainty -- leads to more robust policy learning.by Final Doshi.S.M

Optimal trajectory management for aircraft descent operations subject to time constraints

The growth in traffic increased the pressure on the environmental sustainability of air transport. In this context, many research effort has been devoted to minimise the environmental impact in the different phases of flight. Continuous descent operations, ideally performed with the engines at idle from the cruise altitude to right before landing, have shown to reduce fuel, noise nuisance and gaseous emissions if compared to conventional descents. However, this type of operations suffer from a well known drawback: the loss of predictability from the air traffic control point of view in terms of overfly times at the different waypoints of the route. Due to this loss of predictability, air traffic controllers require large separation buffers, thus reducing the capacity of the airport. Previous works investigating this issue showed that the ability to meet a controlled time of arrival at a metering fix could enable continuous descent operations while simultaneously maintaining airport throughput. In this context, the planning and guidance functions of state-of-the-art flight management systems need to be modernised. On-board trajectory planners capable to generate an optimal trajectory plan satisfying time constraints introduced during the flight are seldom, mainly because the real-time optimisation of aircraft trajectories is still elusive. Furthermore, the time scale and spatial resolution of the wind forecasts used by these trajectory planners are far from being adequate to generate accurate flight time predictions. Finally, there exist guidance strategies capable to accurately comply with time constraints enforced at a certain fix in the trajectory plan, yet they are not specifically designed to minimise the environmental impact. This PhD thesis aims at investigating fast optimisation techniques to enable real-time updates of the optimal trajectory plan subject to time constraints during the descent; wind networking concepts to generate more accurate and up-to-date wind forecasts and, consequently, time predictions; and more robust an efficient guidance strategies to reduce the environmental impact at the maximum extent while complying with the time constraints of the trajectory plan. First, the feasible time window at a metering fix that could be achieved during a descent requiring neither thrust nor speed brakes usage is quantified as a function of the aircraft states (altitude, distance to the metering fix and airspeed), aiming to assess the feasibility of guidance strategies that take advantage of time and energy management concepts. Then, the performance of four of these guidance strategies is compared in terms of environmental impact mitigation and ability to satisfy operational constraints. Results from the comparison reveal that model predictive control, a strategy based on a frequent re-calculation of the optimal trajectory plan during the execution of the descent, is the most robust in terms of energy and time deviation at the metering fix, providing at the same time excellent environmental impact mitigation figures. However, the execution time required to solve a rigorous trajectory optimisation problem at each re-calculation instant remains a critical limitation for practical applications. In order to address this issue, a variant of the model predictive control strategy that allows for fast updates of the optimal trajectory plan based on parametric sensitivities is proposed, which shows analogous results yet halving the time needed to update the optimal trajectory plan. Finally, the potential benefits of using wind observations broadcast by nearby aircraft to reconstruct the wind profile downstream right before updating the optimal trajectory plan when using model predictive control is also investigated. Promising results show that the combination of model predictive control with wind networking concepts could enable optimal descents without degrading the capacity of the airport.El creixement del trànsit ha augmentat la pressió sobre la sostenibilitat ambiental del transport aeri. En aquest àmbit s'han dedicat molts esforços en recerca per reduir l'impacte ambiental en les diferents fases del vol. Les operacions de descens continu, en les quals l'aeronau descendeix amb els motors a ralentí des de l'altitud de creuer fins just abans d'aterrar, han demostrat ser una solució atractiva per reduir el combustible, el soroll i les emissions en la fase de descens. Desafortunadament, aquest tipus d'operacions tenen un inconvenient molt important: la pèrdua de predictibilitat des del punt de vista dels controladors de trànsit aeri, en termes de temps de sobrevol als diferents punts de pas de la ruta. Per aquesta raó, els controladors necessiten aplicar més separació entre aeronaus, reduint així la capacitat de l'aeroport. Treballs anteriors han demostrat que si les aeronaus fossin capaces de satisfer restriccions de temps de sobrevol a un o més punts de pas, seria possible implementar operacions de descens continu sense degradar la capacitat de l'aeroport. Malauradament, avui en dia existeixen pocs sistemes de gestió de vol capaços de generar trajectòries òptimes que satisfacin restriccions de temps, principalment perquè l’optimització de trajectòries en temps real continua sent una tasca difícil. A més, la resolució espacial i temporal dels models de vent utilitzades per els planificadors de trajectòria no son suficients per generar prediccions de temps de sobrevol prou fiables. Finalment, les estratègies de guiatge que fins i tot avui en dia permetrien satisfer amb exactitud restriccions de temps de sobrevol, no estan dissenyades específicament per minimitzar l’impacte ambiental. Aquesta tesi té com a objectiu explorar algoritmes de d'optimització ràpids i robustos que permetin actualitzar la trajectòria òptima en temps real durant l'execució del descens, satisfent al mateix temps restriccions de temps de sobrevol; també s'investigaran nous conceptes de que permetin generar models de vent molt exactes a partir d'observacions emeses per aeronaus veïnes; i estratègies de guiatge més intel·ligents que minimitzin l'impacte ambiental de les operacions de descens continu subjectes a restriccions de temps de sobrevol. En primer lloc, es quantifica la finestra de temps disponible al punt on s'aplica la restricció de temps de sobrevol, en funció dels estats de l'aeronau (altitud, velocitat i distància al punt) i assumint que els motors es mantenen ralentí i que no s'utilitzen aerofrens durant tot el descens. Els resultats de l'experiment indiquen que es podrien utilitzar estratègies de guiatge que gestionessin l'energia cinètica i potencial de l'aeronau per satisfer restriccions de temps sense necessitat de gastar més combustible. A continuació, es compararen quatre d'aquestes estratègies. Els resultats d'aquests segon experiment indiquen que el control predictiu, una estratègia que contínuament actualitza la trajectòria òptima durant el descens, es molt robusta en termes d'errors de temps i energia, i que també redueix l'impacte ambiental. Malauradament, es tarda massa a actualitzar la trajectòria òptima cada cop que s’actualitza, fet que limita la implementació d'aquesta estratègia. Per tal d'afrontar aquesta limitació, es proposa una variant que utilitza sensitivitats paramètriques per reduir el temps d'execució a l'hora d'actualitzar la trajectòria òptima, sense degradar significativament la seva exactitud. Finalment, s'investiguen els possibles beneficis d'aprofitar observacions de vent enviades per les aeronaus del volant per millorar el model de vent i, conseqüentment, l'exactitud de la trajectòria calculada. Resultats prometedors demostren que si s’implementés model predictiu com a estratègia de guiatge i les aeronaus cooperessin per compartir observacions de vent, es reduiria l'impacte ambiental sense degradar la capacitat de l'aeroport