A Risk-Averse Preview-based QQ-Learning Algorithm: Application to Highway Driving of Autonomous Vehicles

A risk-averse preview-based $Q$-learning planner is presented for navigation of autonomous vehicles. To this end, the multi-lane road ahead of a vehicle is represented by a finite-state non-stationary Markov decision process (MDP). A risk assessment unit module is then presented that leverages the preview information provided by sensors along with a stochastic reachability module to assign reward values to the MDP states and update them as scenarios develop. A sampling-based risk-averse preview-based $Q$-learning algorithm is finally developed that generates samples using the preview information and reward function to learn risk-averse optimal planning strategies without actual interaction with the environment. The risk factor is imposed on the objective function to avoid fluctuation of the $Q$ values, which can jeopardize the vehicle's safety and/or performance. The overall hybrid automaton model of the system is leveraged to develop a feasibility check unit module that detects unfeasible plans and enables the planner system to proactively react to the changes of the environment. Theoretical results are provided to bound the number of samples required to guarantee $\epsilon$-optimal planning with a high probability. Finally, to verify the efficiency of the presented algorithm, its implementation on highway driving of an autonomous vehicle in a varying traffic density is considered

Clinical Outcomes of Femtosecond Laser-assisted Implantation of 325-Degree Versus 340-Degree Arc Length Intracorneal Ring Segments in Naive Keratoconic Eyes

Purpose: To evaluate and compare clinical outcomes after femtosecond laser-assisted implantation of 325-degree versus 340-degree arc length intracorneal ring segments (ICRS) in eyes with keratoconus (KCN). Methods: In this prospective non-randomized interventional case series, 23 eyes of 21 patients diagnosed with KCN, underwent femtosecond laser-assisted implantation of two types of ICRS, which included a 325-degree ICRS (Group 325) and a 340-degree ICRS (Group 340). The primary outcome measures were uncorrected distance visual acuity (UDVA), and the secondary outcome measures included corrected distance visual acuity (CDVA), sphere, cylinder, mean refractive spherical equivalent (MRSE), keratometry, vectorial change in corneal astigmatism, and the location of maximum keratometry relative to the corneal apex. The study groups were compared using the primary and secondary outcome measures obtained at postoperative months six and 12. Results: Groups 325 and 340 consisted of 10 and 13 eyes, respectively. The two groups were comparable in terms of parameters measured preoperatively. On comparison to the baseline values, both study groups exhibited a significant increase in UDVA and CDVA measured at postoperative month six (Ps < 0.05) and a significant decrease in the sphere, cylinder, spherical equivalent refraction, and keratometry readings measured at postoperative months six and 12 (Ps < 0.05). No significant differences were observed between the two groups in terms of visual, refractive, and keratometric outcomes at any time point. No intraoperative or postoperative complications were observed in any of the study groups. Conclusion: Both the 325-degree ICRS and the 340-degree ICRS effectively and equally improved visual, refractive, and keratometric outcomes in keratoconic eyes

A Convex Optimization Approach for Control of Linear Quadratic Systems with Multiplicative Noise via System Level Synthesis

This paper presents a convex optimization-based solution to the design of state-feedback controllers for solving the linear quadratic regulator (LQR) problem of uncertain discrete-time systems with multiplicative noise. To synthesize a tractable solution, the recently developed system level synthesis (SLS) framework is leveraged. It is shown that SLS shifts the controller synthesis task from the design of a robust controller to the design of the entire set-valued closed-loop system responses. To this end, the closed-loop system response is entirely characterized by probabilistic set-valued maps from the additive noise to control actions and states. A bi-level convex optimization over the achievable set-valued closed-loop responses is then developed to optimize the expected value of the LQR cost against the worst-case closed-loop system response. The solution to this robust optimization problem may be too conservative since it aims at enforcing the design constraints for all possible system realizations. To deal with this issue, the presented optimization problem is next reformulated as a chance-constrained program (CCP) in which the guarantees are not intended in a deterministic sense of satisfaction against all possible closed-loop system responses, but are instead intended in a probabilistic sense of satisfaction against all but a small fraction of the system responses. To approximately solve the CCP without the requirement of knowing the probabilistic description of the uncertainty in the system matrices, the so-called scenario optimization approach is employed, which provides probabilistic guarantees based on a finite number of system realizations and results in a convex optimization program with moderate computational complexity. Finally, numerical simulations are presented to illustrate the theoretical findings

Finite-time Koopman Identifier: A Unified Batch-online Learning Framework for Joint Learning of Koopman Structure and Parameters

In this paper, a unified batch-online learning approach is introduced to learn a linear representation of nonlinear system dynamics using the Koopman operator. The presented system modeling approach leverages a novel incremental Koopman-based update law that retrieves a mini-batch of samples stored in a memory to not only minimizes the instantaneous Koopman operator's identification errors but also the identification errors for the batch of retrieved samples. Discontinuous modifications of gradient flows are presented for the online update law to assure finite-time convergence under easy-to-verify conditions defined on the batch of data. Therefore, this unified online-batch framework allows performing joint sample- and time-domain analysis for converging the Koopman operator's parameters. More specifically, it is shown that if the collected mini-batch of samples guarantees a rank condition, then finite-time guarantee in the time domain can be certified and the settling time depends on the quality of collected samples being reused in the update law. Moreover, the efficiency of the proposed Koopman-based update law is further analyzed by showing that the identification regret in continuous time grows sub-linearly with time. Furthermore, to avoid learning corrupted dynamics due to the selection of an inappropriate set of Koopman observables, a higher-layer meta learner employs a discrete Bayesian optimization algorithm to obtain the best library of observable functions for the operator. Since finite-time convergence of the Koopman model for each set of observable is guaranteed under a rank condition on stored data, the fitness of each set of observables can be obtained based on the identification error on the stored samples in the proposed framework and even without implementing any controller based on the learned system