Stochastic optimization methods for the simultaneous control of parameter-dependent systems

We address the application of stochastic optimization methods for the simultaneous control of parameter-dependent systems. In particular, we focus on the classical Stochastic Gradient Descent (SGD) approach of Robbins and Monro, and on the recently developed Continuous Stochastic Gradient (CSG) algorithm. We consider the problem of computing simultaneous controls through the minimization of a cost functional defined as the superposition of individual costs for each realization of the system. We compare the performances of these stochastic approaches, in terms of their computational complexity, with those of the more classical Gradient Descent (GD) and Conjugate Gradient (CG) algorithms, and we discuss the advantages and disadvantages of each methodology. In agreement with well-established results in the machine learning context, we show how the SGD and CSG algorithms can significantly reduce the computational burden when treating control problems depending on a large amount of parameters. This is corroborated by numerical experiments

Towards Informed Exploration for Deep Reinforcement Learning

In this thesis, we discuss various techniques for improving exploration for deep reinforcement learning. We begin with a brief review of reinforcement learning (RL) and the fundamental v.s. exploitation trade-off. Then we review how deep RL has improved upon classical and summarize six categories of the latest exploration methods for deep RL, in the order increasing usage of prior information. We then explore representative works in three categories discuss their strengths and weaknesses. The first category, represented by Soft Q-learning, uses regularization to encourage exploration. The second category, represented by count-based via hashing, maps states to hash codes for counting and assigns higher exploration to less-encountered states. The third category utilizes hierarchy and is represented by modular architecture for RL agents to play StarCraft II. Finally, we conclude that exploration by prior knowledge is a promising research direction and suggest topics of potentially impact

On the beliefs off the path: equilibrium refinement due to quantal response and level-k

This paper studies the relevance of equilibrium and nonequilibrium explanations of behavior, with respects to equilibrium refinement, as players gain experience. We investigate this experimentally using an incomplete information sequential move game with heterogeneous preferences and multiple perfect equilibria. Only the limit point of quantal response (the limiting logit equilibrium), and alternatively that of level-k reasoning (extensive form rationalizability), restricts beliefs off the equilibrium path. Both concepts converge to the same unique equilibrium, but the predictions differ prior to convergence. We show that with experience of repeated play in relatively constant environments, subjects approach equilibrium via the quantal response learning path. With experience spanning also across relatively novel environments, though, level-k reasoning tends to dominate

The role of re-appropriation in open design : a case study on how openness in higher education for industrial design engineering can trigger global discussions on the theme of urban gardening

This case study explores the opportunities for students of Industrial Design Engineering to engage with direct and indirect stakeholders by making their design process and results into open-ended Designed Solutions. The reported case study involved 47 students during a two-weeks intensive course on the topic of urban gardening. Observations were collected during three distinctive phases: the co-design phase, the creation of an Open Design and the sharing of these design solutions on the online platform Instructables.com. The open sharing of local solutions triggered more global discussions, based on several types of feedbacks: from simple questions to reference to existing works and from suggestions to critiques. Also some examples of re-appropriation of the designed solutions were reported. These feedbacks show the possibilities for students to have a global vision on their local solutions, confronting them with a wider and more diverse audience. The case study shows on the other hand the difficulty in keeping students engaged in this global discussion, considering how after a few weeks the online discussions dropped to an almost complete silence. It is also impossible with such online platforms to follow the re-appropriation cycles, losing the possibility of exploring the new local context were the replication / modification of the designed product occurred. The course’s focus on Open Design is interesting both under the design and educational points of view. It implies a deep change in the teaching approach and learning attitude of students, allowing unknown peers to take part of the design process and fostering a global discussion starting from unique and local solutions