Improving sample efficiency and multi-agent communication in RL-based train rescheduling

We present preliminary results from our sixth placed entry to the Flatland international competition for train rescheduling, including two improvements for optimized reinforcement learning (RL) training efficiency, and two hypotheses with respect to the prospect of deep RL for complex real-world control tasks: first, that current state of the art policy gradient methods seem inappropriate in the domain of high-consequence environments; second, that learning explicit communication actions (an emerging machine-to-machine language, so to speak) might offer a remedy. These hypotheses need to be confirmed by future work. If confirmed, they hold promises with respect to optimizing highly efficient logistics ecosystems like the Swiss Federal Railways railway network

Color Processing in Zebrafish Retina

Zebrafish (Danio rerio) is a model organism for vertebrate developmental processes and, through a variety of mutant and transgenic lines, various diseases and their complications. Some of these diseases relate to proper function of the visual system. In the US, the National Eye Institute indicates >140 million people over the age of 40 have some form of visual impairment. The causes of the impairments range from refractive error to cataract, diabetic retinopathy and glaucoma, plus heritable diseases such as retinitis pigmentosa and color vision deficits. Most impairments directly affect the retina, the nervous tissue at the back of the eye. Zebrafish with long or short-wavelength color blindness, altered retinal anatomy due to hyperglycemia, high intraocular pressure, and reduced pigment epithelium are all used, and directly applicable, to study how these symptoms affect visual function. However, many published reports describe only molecular/anatomical/structural changes or behavioral deficits. Recent work in zebrafish has documented physiological responses of the different cell types to colored (spectral) light stimuli, indicating a complex level of information processing and color vision in this species. The purpose of this review article is to consolidate published morphological and physiological data from different cells to describe how zebrafish retina is capable of complex visual processing. This information is compared to findings in other vertebrates and relevance to disorders affecting color processing is discussed

Combining reinforcement learning with supervised deep learning for neural active scene understanding

Awarded with the Dr. Waldemar Jucker award 2020 of the GSTWhile vision in living beings is an active process where image acquisition and classification are intertwined to gradually refine perception, much of today’s computer vision is build on the inferior paradigm of episodic classification of i.i.d. samples. We aim at improved scene understanding for robots by taking the sequential nature of seeing over time into account. We present a supervised multi-task approach to answer questions about different aspects of a scene such as the relationship between objects, their quantity or the their relative positions to the camera. For each question, we train a different output head which operates on input from one shared recurrent convolutional neural network that accumulates information over time steps. In parallel, we train an additional output head using reinforcement learning (RL) that uses the reduction in cumulative loss from the supervised heads as reward signal. It thereby learns to gradually improve the prediction confidence of e.g. partially occluded objects by moving the camera to a more favourable angle with respect to these objects. We present preliminary results on simulated RGB-D image sequences that show superior performance of our RL-based approach in answering questions quicker and more accurately than using static or random camera movement

Real-Time Optical Coherence Tomography Controlled Microsecond Laser Retinal Microsurgery: First In-vivo Results

Reliable mild photocoagulation and selective retina therapy (SRT) selectively damaging the retinal pigment epithelium (RPE) while sparing the neuroretina, the photoreceptors as well as the choroid are highly demanded. However, due to the inter- and intraindividual variability of RPE and choroidal absorption, optical microsurgery requires reliable real-time laser dosing to prevent unwanted overexposure and extended damage of the neuroretina. In this experiment optical coherence tomography (OCT) was implemented to detect minimal damage, and a laser feedback control algorithm was used for real-time dosing. For the first time in-vivo experiments on rabbits were performed with microsecond laser pulses of varying duration. Pigment rabbit eyes (n=6) were exposed to laser pulses of 4, 8, 12, and 20 μs in duration (wavelength, 532 nm; ramp-mode, maximum 15 pulses; repetition rate, 100 Hz). Therefore, a system with a scanning laser ophthalmoscope and spectral-domain OCT (Heidelberg Engineering) extended with a prototype laser (Meridian Medical) was used. For each laser lesion, the increasing ramp’s end energy was individually controlled in real-time using OCT dosimetry (central wavelength, 870 nm; scan rate, 80 kHz). Within 1 hour after irradiation, retinal changes were assessed with fluorescein angiography (FA), indocyanine green angiography (ICGA), color fundus photography (CFP) and OCT. OCT dosimetry utilizing the control algorithm can interrupt the ramp-mode in real-time for each lesion individually. The preconditioned algorithm enabled treatment with a clearly visible breakdown of the blood-retinal barrier (BRB) according to FA and ICGA imaging and barely visible treatment lesions according to CFP. OCT B-scans through the treated areas provided a first indication of the morphological tissue impact. Preliminary evaluation shows that the algorithm stopped the laser at 4 μs at a ramp end energy of 53 μJ (corresponds to 13/15 pulses), at 8 μs at 68 μJ (5/15 pulses), at 12 μs at 74 μJ (7/15 pulses), and at 20 μs at 100 μJ (1/15 pulses). The novel system with OCT based laser dosing proved to induce minimal visible damage and BRB breakdown in a wide range of pulse durations. The new irradiation scheme and algorithm are being optimized and tested in multiple subjects to further limit unwanted damage and enable pure RPE selective laser microsurgery in real-time. This abstract was presented at the 2022 ARVO Annual Meeting, held in Denver, CO, May 1-4, 2022, and virtually

Controlling the collimation and rotation of hydromagnetic disk winds

(Abriged) We present a comprehensive set of axisymmetric, time-dependent simulations of jets from Keplerian disks whose mass loading as a function of disk radius is systematically changed. For a reasonable model for the density structure and injection speed of the underlying accretion disk, mass loading is determined by the radial structure of the disk's magnetic field structure. We vary this structure by using four different magnetic field configurations, ranging from the "potential" configuration (Ouyed&Pudritz 1997), to the increasingly more steeply falling Blandford&Payne (1982) and Pelletier&Pudritz (1992) models, and ending with a quite steeply raked configuration that bears similarities to the Shu X-wind model. We find that the radial distribution of the mass load has a profound effect on both the rotational profile of the underlying jet as well as the degree of collimation of its outflow velocity and magnetic field lines. We show analytically, and confirm by our simulations, that the collimation of a jet depends on its radial current distribution, which in turn is prescribed by the mass load. Models with steeply descending mass loads have strong toroidal fields, and these collimate to cylinders (this includes the Ouyed-Pudritz and Blandford-Payne outflows). On the other hand, the more gradually descending mass load profiles (the PP92 and monopolar distributions) have weaker toroidal fields, and these result in wide-angle outflows with parabolic collimation. We also present detailed structural information about jets such as their radial profiles of jet density, toroidal magnetic field, and poloidal jet speed, as well as an analysis of the bulk energetics of our different simulations.Comment: 20 journal pages, including 9 figures. Submitted to MNRA

Evolution of the nucleus

Under a Creative Commons license.The nucleus represents a major evolutionary transition. As a consequence of separating translation from transcription many new functions arose, which likely contributed to the remarkable success of eukaryotic cells. Here we will consider what has recently emerged on the evolutionary histories of several key aspects of nuclear biology; the nuclear pore complex, the lamina, centrosomes and evidence for prokaryotic origins of relevant players.Work in our laboratories was supported by the following agencies, and which is gratefully acknowledged; MRC and Wellcome Trust (MR/K008749/1 and 090007/Z/09/Z respectively, to MCF), C2A Junta de Andalucia to DPD and DFG GR1642/4-1 to RG.Open Access funded by Wellcome Trust.Peer Reviewe

Siponimod vs placebo in active secondary progressive multiple sclerosis: a post hoc analysis from the phase 3 EXPAND study.

BACKGROUND Siponimod is a sphingosine 1-phosphate receptor modulator approved for active secondary progressive multiple sclerosis (aSPMS) in most countries; however, phase 3 EXPAND study data are from an SPMS population with/without disease activity. A need exists to characterize efficacy/safety of siponimod in aSPMS. METHODS Post hoc analysis of participants with aSPMS (≥ 1 relapse in 2 years before study and/or ≥ 1 T1 gadolinium-enhancing [Gd +] magnetic resonance imaging [MRI] lesions at baseline) receiving oral siponimod (2 mg/day) or placebo for up to 3 years in EXPAND. ENDPOINTS 3-month/6-month confirmed disability progression (3mCDP/6mCDP); 3-month confirmed ≥ 20% worsening in Timed 25-Foot Walk (T25FW); 6-month confirmed improvement/worsening in Symbol Digit Modalities Test (SDMT) scores (≥ 4-point change); T2 lesion volume (T2LV) change from baseline; number of T1 Gd + lesions baseline-month 24; number of new/enlarging (N/E) T2 lesions over all visits. RESULTS Data from 779 participants with aSPMS were analysed. Siponimod reduced risk of 3mCDP/6mCDP vs placebo (by 31%/37%, respectively; p < 0.01); there was no significant effect on T25FW. Siponimod increased likelihood of 6-month confirmed SDMT improvement vs placebo (by 62%; p = 0.007) and reduced risk of 6-month confirmed SDMT worsening (by 27%; p = 0.060). Siponimod was associated with less increase in T2LV (1316.3 vs 13.3 mm3; p < 0.0001), and fewer T1 Gd + and N/E T2 lesions than placebo (85% and 80% reductions, respectively; p < 0.0001). CONCLUSIONS In aSPMS, siponimod reduced risk of disability progression and was associated with benefits on cognition and MRI outcomes vs placebo. TRIAL REGISTRATION ClinicalTrials.gov number: NCT01665144

Insulin resistance causes inflammation in adipose tissue

Obesity is a major risk factor for insulin resistance and type 2 diabetes. In adipose tissue, obesity-mediated insulin resistance correlates with the accumulation of proinflammatory macrophages and inflammation. However, the causal relationship of these events is unclear. Here, we report that obesity-induced insulin resistance in mice precedes macrophage accumulation and inflammation in adipose tissue. Using a mouse model that combines genetically induced, adipose-specific insulin resistance (mTORC2-knockout) and diet-induced obesity, we found that insulin resistance causes local accumulation of proinflammatory macrophages. Mechanistically, insulin resistance in adipocytes results in production of the chemokine monocyte chemoattractant protein 1 (MCP1), which recruits monocytes and activates proinflammatory macrophages. Finally, insulin resistance (high homeostatic model assessment of insulin resistance [HOMA-IR]) correlated with reduced insulin/mTORC2 signaling and elevated MCP1 production in visceral adipose tissue from obese human subjects. Our findings suggest that insulin resistance in adipose tissue leads to inflammation rather than vice versa