Learning to Search in Reinforcement Learning

In this thesis, we investigate the use of search based algorithms with deep neural networks to tackle a wide range of problems ranging from board games to video games and beyond. Drawing inspiration from AlphaGo, the first computer program to achieve superhuman performance in the game of Go, we developed a new algorithm AlphaZero. AlphaZero is a general reinforcement learning algorithm that combines deep neural networks with a Monte Carlo Tree search for planning and learning. Starting completely from scratch, without any prior human knowledge beyond the basic rules of the game, AlphaZero managed to achieve superhuman performance in Go, chess and shogi. Subsequently, building upon the success of AlphaZero, we investigated ways to extend our methods to problems in which the rules are not known or cannot be hand-coded. This line of work led to the development of MuZero, a model-based reinforcement learning agent that builds a deterministic internal model of the world and uses it to construct plans in its imagination. We applied our method to Go, chess, shogi and the classic Atari suite of video-games, achieving superhuman performance. MuZero is the first RL algorithm to master a variety of both canonical challenges for high performance planning and visually complex problems using the same principles. Finally, we describe Stochastic MuZero, a general agent that extends the applicability of MuZero to highly stochastic environments. We show that our method achieves superhuman performance in stochastic domains such as backgammon and the classic game of 2048 while matching the performance of MuZero in deterministic ones like Go

Playing Atari with Deep Reinforcement Learning

We present the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning. The model is a convolutional neural network, trained with a variant of Q-learning, whose input is raw pixels and whose output is a value function estimating future rewards. We apply our method to seven Atari 2600 games from the Arcade Learning Environment, with no adjustment of the architecture or learning algorithm. We find that it outperforms all previous approaches on six of the games and surpasses a human expert on three of them.Comment: NIPS Deep Learning Workshop 201

Methods, transparency and reporting of clinical trials in orthodontics and periodontics

Objective: The aim of this study was to explore the methods, reporting and transparency of clinical trials in orthodontics and compare them to the field of periodontics, as a standard within dentistry. Design/setting: Cross-sectional bibliographic study. Methods: A total of 300 trials published in 2017-2018 and evenly distributed in orthodontics and periodontics were selected, assessed and analysed statistically to explore key aspects of the conduct and reporting of orthodontic clinical trials compared to trials in periodontics. Results: Several aspects are often neglected in orthodontic and periodontic trials and could be improved upon, including use of statistical expertise (22.3% of assessed trials), blinding of outcome assessors (62.3%), prospective trial registration (12.0%), adequate sample size calculation (35.7%), adherence to CONSORT (14.3%) and open data sharing (4.3%). The prevalence of statistically significant findings among orthodontic and periodontic trials was 62.3%, which was significantly associated with several methodological traits like statistician involvement (odds ratio [OR] = 0.5; 95% confidence interval [CI] = 0.3-0.9), blind outcome assessor (OR = 0.5; 95% CI = 0.2-1.0), lack of prospective trial registration (OR = 2.8; 95% CI = 1.3-5.9) and non-adherence to CONSORT (OR = 4.5; 95% CI = 1.3-15.8). Conclusions: Although trials in orthodontics seem to be significantly worse compared to periodontics in aspects like trial registration, adherence to CONSORT and declaration of competing interests or financial support, their methods do seem to have improved considerably in recent years

Καθιέρωση και χαρακτηρισμός υδρογέλης από αποκυτταρωμένο έντερο ποντικού

Το Matrigel είναι μια δημοφιλής επιλογή για την καλλιέργεια οργανοειδών λόγω της αποτελεσματικότητάς του. Ωστόσο, εμπεριέχει αρκετούς περιορισμούς. Η διακύμανση της παρτίδας, οι ανησυχίες για την ασφάλεια και η απουσία ειδικών για τον ιστό βιοχημικών παραγόντων εξωκυτταρικής μήτρας είναι κρίσιμα ζητήματα που πρέπει να ληφθούν υπόψη. Κατά τη διάρκεια της διατριβής μου, θέλαμε να διαπιστώσουμε εάν ο αποκυτταρωμένος εντερικός ιστός ποντικού μπορεί να χρησιμοποιηθεί ως μήτρα για καλλιέργεια μεσεγχυματικών κυττάρων εντέρου. Για το σκοπό αυτό, δοκιμάσαμε την αποτελεσματικότητα του πρωτοκόλλου για την αποκυτταροποίηση του παχέος εντέρου των ποντικών. Η διαδικασία αποκυτταροποίησης ήταν επιτυχής. Επαληθεύτηκε από μια σημαντική μείωση στη συγκέντρωση του DNA και στον αριθμό των κυττάρων. Η ομοεστιακή απεικόνιση αποκάλυψε την απουσία κυττάρων και την παρουσία δικτύων ColIV που οριοθετούν τις δομές της κρύπτης. Η ανάλυση HPLC-MS/MS αποκάλυψε την πρωτεϊνική σύνθεση του αποκυτταρωμένης εξωκυτταρικής μήτρας, η οποία έχει αξιοσημείωτη ομοιότητα με μια δημόσια διαθέσιμη μητροσωμική υπογραφή φυσιολογικού παχέος εντέρου ποντικού. Περαιτέρω ανάλυση της πρωτεϊνικής σύνθεσης αποκυτταρομένου ιστού παχέος εντέρου από ποντίκια που έλαβαν DSS, κατά τη διάρκεια οξείας φλεγμονής και αναγέννησης έδειξε σημαντικές διαφορές μεταξύ των διαφορετικών χρονικών σημείων, ειδικά σε πρωτεΐνες που σχετίζονται με την εξωκυτταρική μήτρα. Τα SFRP1 και COL18A1 επιλέχθηκαν για να σεσχυετιστούν τα επιπέδα πρωτεΐνης εξωκυτταρικής μήτρας και της γονιδιακής έκφρασης του ιστού με qRT-PCR. Τέλος, η ανάπτυξη υδρογέλης από αποκυτταρωμένο εντερικό ιστό ήταν επιτυχής, αλλά χρειάζεται περαιτέρω έρευνα για να διαπιστωθεί η αποτελεσματικότητα και η ασφάλειά του.Matrigel is a popular choice for growing organoids due to its effectiveness. However, it is not without limitations. Batch variation, safety concerns, and the absence of tissue-specific biochemical ECM factors are all critical issues that should be considered. During my dissertation thesis we set out to establish whether mouse decellularized intestinal tissue can be used as a matrix for IMC culture. To that end, we tested the efficiency of the protocol for the decellularization of the colon of mice. The decellularization process was successful, verified by a significant decrease in DNA concentration and number of cells. Confocal imaging revealed the absence of cells and the presence of ColIV networks delineating crypt structures. HPLC-MS/MS analysis uncovered the protein composition of the decellularized ECM, which has remarkable similarity with a publicly available matrisomal signature of normal mouse colon. Further analysis of the protein composition of decellularized colon tissue from mice treated with DSS during acute inflammation and regeneration showed significant differences between the different timepoints, especially in ECM related proteins. SFRP1 and COL18A1 were selected to correlate between ECM protein levels and tissue gene expression by qRT-PCR. Finally, the development of hydrogels from decellularized intestinal tissue was successful, but more work is needed to establish its efficiency and safety