Understanding the Complexity Gains of Single-Task RL with a Curriculum

Reinforcement learning (RL) problems can be challenging without well-shaped rewards. Prior work on provably efficient RL methods generally proposes to address this issue with dedicated exploration strategies. However, another way to tackle this challenge is to reformulate it as a multi-task RL problem, where the task space contains not only the challenging task of interest but also easier tasks that implicitly function as a curriculum. Such a reformulation opens up the possibility of running existing multi-task RL methods as a more efficient alternative to solving a single challenging task from scratch. In this work, we provide a theoretical framework that reformulates a single-task RL problem as a multi-task RL problem defined by a curriculum. Under mild regularity conditions on the curriculum, we show that sequentially solving each task in the multi-task RL problem is more computationally efficient than solving the original single-task problem, without any explicit exploration bonuses or other exploration strategies. We also show that our theoretical insights can be translated into an effective practical learning algorithm that can accelerate curriculum learning on simulated robotic tasks

Embryonic Pattern Scaling Achieved by Oppositely Directed Morphogen Gradients

Morphogens are proteins, often produced in a localised region, whose concentrations spatially demarcate regions of differing gene expression in developing embryos. The boundaries of expression must be set accurately and in proportion to the size of the one-dimensional developing field; this cannot be accomplished by a single gradient. Here, we show how a pair of morphogens produced at opposite ends of a developing field can solve the pattern-scaling problem. In the most promising scenario, the morphogens effectively interact according to the annihilation reaction $A+B\to\emptyset$ and the switch occurs according to the absolute concentration of $A$ or $B$. In this case embryonic markers across the entire developing field scale approximately with system size; this cannot be achieved with a pair of non-interacting gradients that combinatorially regulate downstream genes. This scaling occurs in a window of developing-field sizes centred at a few times the morphogen decay length.Comment: 24 pages; 11 figures; uses iopar

Sorafenib dose escalation is not uniformly associated with blood pressure elevations in normotensive patients with advanced malignancies.

Hypertension after treatment with vascular endothelial growth factor (VEGF) receptor inhibitors is associated with superior treatment outcomes for advanced cancer patients. To determine whether increased sorafenib doses cause incremental increases in blood pressure (BP), we measured 12-h ambulatory BP in 41 normotensive advanced solid tumor patients in a randomized dose-escalation study. After 7 days' treatment (400 mg b.i.d.), mean diastolic BP (DBP) increased in both study groups. After dose escalation, group A (400 mg t.i.d.) had marginally significant further increase in 12-h mean DBP (P = 0.053), but group B (600 mg b.i.d.) did not achieve statistically significant increases (P = 0.25). Within groups, individuals varied in BP response to sorafenib dose escalation, but these differences did not correlate with changes in steady-state plasma sorafenib concentrations. These findings in normotensive patients suggest BP is a complex pharmacodynamic biomarker of VEGF inhibition. Patients have intrinsic differences in sensitivity to sorafenib's BP-elevating effects

Treatment-resistant pediatric giant prolactinoma and multiple endocrine neoplasia type 1

Background Pediatric pituitary adenomas are rare, accounting for <3 % of all childhood intracranial tumors, the majority of which are prolactinomas. Consequently, they are often misdiagnosed as other suprasellar masses such as craniopharyngiomas in this age group. Whilst guidelines exist for the treatment of adult prolactinomas, the management of childhood presentations of these benign tumors is less clear, particularly when dopamine agonist therapy fails. Given their rarity, childhood-onset pituitary adenomas are more likely to be associated with a variety of genetic syndromes, the commonest being multiple endocrine neoplasia type 1 (MEN-1). Case description We present a case of an early-onset, treatment-resistant giant prolactinoma occurring in an 11-year-old peripubertal boy that was initially sensitive, but subsequently highly resistant to dopamine agonist therapy, ultimately requiring multiple surgical debulking procedures and proton beam irradiation. Our patient is now left with long-term tumor- and treatment-related neuroendocrine morbidities including blindness and panhypopituitarism. Only after multiple consultations and clinical data gained from 20-year-old medical records was a complex, intergenerationally consanguineous family history revealed, compatible with MEN-1, with a splice site mutation (c.784-9G > A) being eventually identified in intron 4 of the MEN1 gene, potentially explaining the difficulties in management of this tumor. Genetic counseling and screening has now been offered to the wider family. Conclusions This case emphasizes the need to consider pituitary adenomas in the differential diagnosis of all pediatric suprasellar tumors by careful endocrine assessment and measurement of at least a serum prolactin concentration. It also highlights the lack of evidence for the optimal management of pediatric drug-resistant prolactinomas. Finally, the case we describe demonstrates the importance of a detailed family history and the role of genetic testing for MEN1 and AIP mutations in all cases of pediatric pituitary adenoma

Using Design of Experiments in Finite Element Modeling to Identify Critical Variables for Laser Powder Bed Fusion

Input of accurate material and simulation parameters is critical for accurate predictions in Laser Powder Bed Fusion (L-PBF) Finite Element Analysis (FEA). It is challenging and resource consuming to run experiments that measure and control all possible material properties and process parameters. In this research, we developed a 3-dimensional thermal L-PBF FEA model for a single track laser scan on one layer of metal powder above a solid metal substrate. We applied a design of experiments (DOE) approach which varies simulation parameters to identify critical variables in L-PBF. DOE is an exploratory tool for examining a large number of factors and alternative modeling approaches. It also determines which approaches can best predict L-PBF process performance.Mechanical Engineerin