Ranked Reward: Enabling Self-Play Reinforcement Learning for Combinatorial Optimization

Adversarial self-play in two-player games has delivered impressive results when used with reinforcement learning algorithms that combine deep neural networks and tree search. Algorithms like AlphaZero and Expert Iteration learn tabula-rasa, producing highly informative training data on the fly. However, the self-play training strategy is not directly applicable to single-player games. Recently, several practically important combinatorial optimisation problems, such as the travelling salesman problem and the bin packing problem, have been reformulated as reinforcement learning problems, increasing the importance of enabling the benefits of self-play beyond two-player games. We present the Ranked Reward (R2) algorithm which accomplishes this by ranking the rewards obtained by a single agent over multiple games to create a relative performance metric. Results from applying the R2 algorithm to instances of a two-dimensional and three-dimensional bin packing problems show that it outperforms generic Monte Carlo tree search, heuristic algorithms and integer programming solvers. We also present an analysis of the ranked reward mechanism, in particular, the effects of problem instances with varying difficulty and different ranking thresholds

Celiac disease: a model autoimmune disease with gene therapy applications

Gene therapy (GT) is still at the 'experimental' stage and some recent setbacks have cooled the potential use of this therapeutic tool even in life-threatening conditions. However, this therapeutic approach has a potential, which is not limited to disease for which we have not other option. There are increasing evidence that GT will be soon used in diseases that are not life threatening. One group of diseases that can benefit from GT is the autoimmune one. Several experimental animal models have indicated the efficacy (proof of principle) of GT. In the present review, we have addressed the possibility that even extremely benign autoimmune-like diseases such as Celiac Disease (CD) might one day profit from this type of therapy. We further point that in conditions such as CD, where the trigger is well known and the pathogenic cascade is relatively well defined, a situation not common in autoimmunity, we can even have a better situation where to explore and use GT to control disease initiation and progression. Once the risks that are still intrinsic to GT will have been reduced the therapeutic options we outline in the present review might not appear too far from reality