Control of a Mixed Autonomy Signalised Urban Intersection: An Action-Delayed Reinforcement Learning Approach

We consider a mixed autonomy scenario where the traffic intersection controller decides whether the traffic light will be green or red at each lane for multiple traffic-light blocks. The objective of the traffic intersection controller is to minimize the queue length at each lane and maximize the outflow of vehicles over each block. We consider that the traffic intersection controller informs the autonomous vehicle (AV) whether the traffic light will be green or red for the future traffic-light block. Thus, the AV can adapt its dynamics by solving an optimal control problem. We model the decision process of the traffic intersection controller as a deterministic delay Markov decision process owing to the delayed action by the traffic controller. We propose Reinforcement-learning based algorithm to obtain the optimal policy. We show - empirically - that our algorithm converges and reduces the energy costs of AVs drastically as the traffic controller communicates with the AVs.Comment: Accepted for Publication at 24th IEEE International Conference on Intelligent Transportation (ITSC'2021

Event-Triggered Action-Delayed Reinforcement Learning Control of a Mixed Autonomy Signalised Urban Intersection

We propose an event-triggered framework for deciding the traffic light at each lane in a mixed autonomy scenario. We deploy the decision after a suitable delay, and events are triggered based on the satisfaction of a predefined set of conditions. We design the trigger conditions and the delay to increase the vehicles’ throughput. This way, we achieve full exploitation of autonomous vehicles (AVs) potential. The ultimate goal is to obtain vehicle-flows led by AVs at the head. We formulate the decision process of the traffic intersection controller as a deterministic delayed Markov decision process, i.e., the action implementation and evaluation are delayed. We propose a Reinforcement Learning based model-free algorithm to obtain the optimal policy. We show - by simulations - that our algorithm converges, and significantly reduces the average wait-time and the queues length as the fraction of the AVs increases. Our algorithm outperforms our previous work [1] by a quite significant amount

Model-Free Plant Tuning

Given a static plant described by a differentiable input-output function, which is completely unknown, but whose Jacobian takes values in a known polytope in the matrix space, this paper considers the problem of tuning (i.e., driving to a desired value) the output, by suitably choosing the input. It is shown that, if the polytope is robustly nonsingular (or has full rank, in the nonsquare case), then a suitable tuning scheme drives the output to the desired point. The proof exploits a Lyapunov-like function and applies a well-known game-theoretic result, concerning the existence of a saddle point for a min-max zero-sum game. When the plant output is represented in an implicit form, it is shown that the same result can be obtained, resorting to a different Lyapunov-like function. The case in which proper input or output constraints must be enforced during the transient is considered as well. Some application examples are proposed to show the effectiveness of the approach

Model Predictive Control for Temperature Regulation of Professional Ovens

We apply the model predictive control (MPC) strategy in an industrial setting, specifically for controlling the temperature of Combi Oven Professional Appliances. The proposed method takes into account input and output constraints, as well as the presence of multiple sources of disturbance. The workflow includes identifying and validating a model of the cell temperature and incorporating disturbance models. MPC is implemented using a state-space formulation. The proposed method shows significant energy saving and tracking error reduction with respect to the current oven control; its effectiveness has been demonstrated through several tests carried out on a professional oven

Model-free feedback control synthesis from expert demonstration

We show how it is possible to synthesize a stabilizing feedback control, in the complete absence of a model, starting from the open-loop control generated by an expert operator, capable of driving a system to a specific set-point. We assume that the system is linear and discrete time. We propose two different controls: a linear dynamic and a static, piecewise linear, one. We show the performance of the proposed controllers on a ship steering problem

A convex programming approach to the inverse kinematics problem for manipulators under constraints

Abstract We propose a novel approach to the problem of inverse kinematics for possibly redundant planar manipulators. We show that, by considering the joints as point masses in a fictitious gravity field, and by adding proper constraints to take into account the length of the links, the kinematic inversion may be cast as a convex programming problem. Convex constraints in the decision variables (in particular, linear constraints in the workspace) are easily managed with the proposed approach. We also show how to exploit the idea for avoiding obstacles while tracking a reference end-effector trajectory and discuss how to extend the results to some kinds of non-planar manipulators. Simulation results are reported, showing the effectiveness of the approach

Quality of images with toric intraocular lenses

Purpose: To objectively evaluate the image quality obtained with toric intraocular lenses (IOLs) when misaligned from the intended axis. Setting: University Eye Clinic and the Department of Industrial and Information Engineering, University of Trieste, Trieste, Italy. Design: Experimental study. Methods: An experimental optoelectronic test bench was created. It consisted of a high-resolution monitor to project target images and an artificial eye. The system simulates the optical and geometric characteristics of the human eye with an implanted toric IOL. A 3.00 diopters corneal astigmatism was simulated. Images reproduced by the optical system were captured according to different IOL axis positions. The quality of each image was analyzed using the visual information fidelity (VIF) criterion. The VIF reduction was calculated at each IOL rotational step. Results: A 5-degree IOL axis rotation from the intended position determined a decay in the image quality of 7.03%. Ten degrees of IOL rotation caused an 11.09% decay of relative VIF value. For a 30-degree rotation, the VIF decay value was 45.85%. Finally, the image decay with no toric correction was 56.70%. Conclusions: The more the objective quality of the image decays progressively, the further the axis of the IOL is rotated from its intended position. The reduction in image quality obtained after 30 degrees of toric IOL rotation was less than 50% and after 45 degrees, the image quality was the same as that of no toric correction

Free-electron laser spectrum evaluation and automatic optimization

The radiation generated by a seeded free-electron laser (FEL) is characterized by a high temporal coherence, which is close to the Fourier limit in the ideal case. The setup and optimization of a FEL is a non-trivial and challenging operation. This is due to the plethora of highly sensitive machine parameters and to the complex correlations between them. The fine tuning of the FEL process is normally supervised by physicists and is carried out by scanning various parameters with the aim of optimizing the spectrum of the emitted pulses in terms of intensity and line-width. In this article we introduce a novel quantitative method for the evaluation of the FEL spectrum via a quality index. Moreover, we investigate the possibility of optimization of the FEL parameters using this index as the objective function of an automatic procedure. We also present the results of the preliminary tests performed in the FERMI FEL focused on the effectiveness and ability of the automatic procedure to assist in the task of machine tuning and optimization

A control system for preventing cavitation of centrifugal pumps

Cavitation is a well-known phenomenon that may occur, among other turbo-machines, in centrifugal pumps and can result in severe damage of both the pump and the whole hydraulic system. There are situations in which, in principle, the cavitation could be avoided by detecting the condition of incipient cavitation, and changing slightly the working point of the whole system in order to move away from that condition. In the present paper two simple closed-loop control strategies are implemented, acting on the pump's rotational speed and fed by the measurements of a set of inertial sensors. In particular, the research is focused on a centrifugal pump normally employed in hydraulic systems. The pump operates in a dedicated test rig, where cavitation can be induced by acting on a reservoir's pressure. Three accelerometers are installed in the pump body along three orthogonal axes. An extensive set of experiments has been carried out at different flow rates and a number of signals' features both in the time domain and in the frequency domain have been considered as indicators of incipient cavitation. The amount of energy of the signal captured by the accelerometer in the component orthogonal to the flow direction, in the band from 10 to 12.8 kHz, demonstrated to be effective in detecting the incipient cavitation, by selecting a proper (condition-dependent) threshold. Therefore, two simple controllers have been designed: the first regulates the speed of the pump, to recover from cavitation, bringing the indicator back to the nominal value, while the second allows to reduce the pump's rotational speed when the cavitation detector indicates the incipient cavitation and restoring the nominal speed when possible. The latter approach is rather general, because the threshold-based detector can be substituted by any detector providing binary output. Experimental results are reported that demonstrate the effectiveness of the approach

Mosaic Images Segmentation using U-net

We consider the task of segmentation of images of mosaics, where the goal is to segment the image in such a way that each region corresponds exactly to one tile of the mosaic. We propose to use a recent deep learning technique based on a kind of convolutional neural networks, called U-net, that proved to be effective in segmentation tasks. Our method includes a preprocessing phase that allows to learn a U-net despite the scarcity of labeled data, which reflects the peculiarity of the task, in which manual annotation is, in general, costly. We experimentally evaluate our method and compare it against the few other methods for mosaic images segmentation using a set of performance indexes, previously proposed for this task, computed using 11 images of real mosaics. In our results, U-net compares favorably with previous methods. Interestingly, the considered methods make errors of different kinds, consistently with the fact that they are based on different assumptions and techniques. This finding suggests that combining different approaches might lead to an even more effective segmentation