Robust Attitude Control of an Agile Aircraft Using Improved Q-Learning

Attitude control of a novel regional truss-braced wing (TBW) aircraft with low stability characteristics is addressed in this paper using Reinforcement Learning (RL). In recent years, RL has been increasingly employed in challenging applications, particularly, autonomous flight control. However, a significant predicament confronting discrete RL algorithms is the dimension limitation of the state-action table and difficulties in defining the elements of the RL environment. To address these issues, in this paper, a detailed mathematical model of the mentioned aircraft is first developed to shape an RL environment. Subsequently, Q-learning, the most prevalent discrete RL algorithm, will be implemented in both the Markov Decision Process (MDP) and Partially Observable Markov Decision Process (POMDP) frameworks to control the longitudinal mode of the proposed aircraft. In order to eliminate residual fluctuations that are a consequence of discrete action selection, and simultaneously track variable pitch angles, a Fuzzy Action Assignment (FAA) method is proposed to generate continuous control commands using the trained optimal Q-table. Accordingly, it will be proved that by defining a comprehensive reward function based on dynamic behavior considerations, along with observing all crucial states (equivalent to satisfying the Markov Property), the air vehicle would be capable of tracking the desired attitude in the presence of different uncertain dynamics including measurement noises, atmospheric disturbances, actuator faults, and model uncertainties where the performance of the introduced control system surpasses a well-tuned Proportional–Integral–Derivative (PID) controller

Lunar Flashlight: Mapping Lunar Surface Volatiles Using a Cubesat

Water ice and other volatiles may be located in the Moon's polar regions, with sufficient quantities for in situ extraction and utilization by future human and robotic missions. Evidence from orbiting spacecraft and the LCROSS impactor suggests the presence of surface and/or nearsurface volatiles, including water ice. These deposits are of interest to human exploration to understand their potential for use by astronauts. Understanding the composition, quantity, distribution, and form of water/H species and other volatiles associated with lunar cold traps is identified as a NASA Strategic Knowledge Gap (SKG) for Human Exploration. These polar volatile deposits could also reveal important information about the delivery of water to the Earth- Moon system, so are of scientific interest. The scientific exploration of the lunar polar regions was one of the key recommendations of the Planetary Science Decadal Survey. In order to address NASA's SKGs, the Advanced Exploration Systems (AES) program selected three lowcost 6-U CubeSat missions for launch as secondary payloads on the first test flight (EM1) of the Space Launch System (SLS) scheduled for 2017. The Lunar Flashlight mission was selected as one of these missions, specifically to address the SKG associated with lunar volatiles. Development of the Lunar Flashlight CubeSat concept leverages JPL's Interplanetary Nano- Spacecraft Pathfinder In Relevant Environment (INSPIRE) mission, MSFC's intimate knowledge of the Space Launch System and EM-1 mission, small business development of solar sail and electric propulsion hardware, and JPL experience with specialized miniature sensors. The goal of Lunar Flashlight is to determine the presence or absence of exposed water ice and its physical state, and map its concentration at the kilometer scale within the permanently shadowed regions of the lunar south pole. After being ejected in cislunar space by SLS, Lunar Flashlight deploys its solar panels and solar sail and maneuvers into a low-energy transfer to lunar orbit. The solar sail and attitude control system work to bring the satellite into an elliptical polar orbit spiraling down to a perilune of 30-10 km above the south pole for data collection. Lunar Flashlight uses its solar sail to shine reflected sunlight into permanently shadowed regions, measuring surface albedo with a four-filter point spectrometer at 1.1, 1.5 1.9, and 2.0 microns. Water ice will be distinguished from dry regolith from these measurements in two ways: 1) spatial variations in absolute reflectance (water ice is much brighter in the continuum channels), and 2) reflectance ratios between absorption and continuum channels. Derived reflectance and reflectance ratios will be mapped onto the lunar surface in order to distinguish the composition of the PSRs from that of the sunlit terrain. Lunar Flashlight enables a low-cost path to in-situ resource utilization (ISRU) by identifying operationally useful deposits (if there are any), which is a game-changing capability for expanded human exploration