Guided Data Augmentation for Offline Reinforcement Learning and Imitation Learning

Learning from demonstration (LfD) is a popular technique that uses expert demonstrations to learn robot control policies. However, the difficulty in acquiring expert-quality demonstrations limits the applicability of LfD methods: real-world data collection is often costly, and the quality of the demonstrations depends greatly on the demonstrator's abilities and safety concerns. A number of works have leveraged data augmentation (DA) to inexpensively generate additional demonstration data, but most DA works generate augmented data in a random fashion and ultimately produce highly suboptimal data. In this work, we propose Guided Data Augmentation (GuDA), a human-guided DA framework that generates expert-quality augmented data. The key insight of GuDA is that while it may be difficult to demonstrate the sequence of actions required to produce expert data, a user can often easily identify when an augmented trajectory segment represents task progress. Thus, the user can impose a series of simple rules on the DA process to automatically generate augmented samples that approximate expert behavior. To extract a policy from GuDA, we use off-the-shelf offline reinforcement learning and behavior cloning algorithms. We evaluate GuDA on a physical robot soccer task as well as simulated D4RL navigation tasks, a simulated autonomous driving task, and a simulated soccer task. Empirically, we find that GuDA enables learning from a small set of potentially suboptimal demonstrations and substantially outperforms a DA strategy that samples augmented data randomly

Image segmentation with implicit color standardization using spatially constrained expectation maximization: Detection of nuclei

Abstract. Color nonstandardness -the propensity for similar objects to exhibit different color properties across images -poses a significant problem in the computerized analysis of histopathology. Though many papers propose means for improving color constancy, the vast majority assume image formation via reflective light instead of light transmission as in microscopy, and thus are inappropriate for histological analysis. Previously, we presented a novel Bayesian color segmentation algorithm for histological images that is highly robust to color nonstandardness; this algorithm employed the expectation maximization (EM) algorithm to dynamically estimate -for each individual image -the probability density functions that describe the colors of salient objects. However, our approach, like most EM-based algorithms, ignored important spatial constraints, such as those modeled by Markov random field (MRFs). Addressing this deficiency, we now present spatially-constrained EM (SCEM), a novel approach for incorporating Markov priors into the EM framework. With respect to our segmentation system, we replace EM with SCEM and then assess its improved ability to segment nuclei in H&E stained histopathology. Segmentation performance is evaluated over seven (nearly) identical sections of gastrointestinal tissue stained using different protocols (simulating severe color nonstandardness). Over this dataset, our system identifies nuclear regions with an area under the receiver operator characteristic curve (AUC) of 0.838. If we disregard spatial constraints, the AUC drops to 0.748

EARLINET: 12-year of aerosol profiling over Europe

EARLINET has been collecting high quality aerosol optical profiles over Europe since 2000. The comparison with automatic collected dataset of aerosol optical depth (AOD) from AERONET and MODIS demonstrates the effectiveness of EARLINET regular measurement schedule for climatological studies. The analysis of optical properties in the local boundary layer indicates that the general decrease of AOD observed by different platforms over Europe in the last decade could be due to the modification of aerosol properties (towards less absorbing and larger particles) in the lower troposphere

Four-dimensional distribution of the 2010 Eyjafjallajökull volcanic cloud over Europe observed by EARLINET

© Author(s) 2013. This work is distributed under the Creative Commons Attribution 3.0 License.The eruption of the Icelandic volcano Eyjafjallaj ökull in April-May 2010 represents a "natural experiment" to study the impact of volcanic emissions on a continental scale. For the first time, quantitative data about the presence, altitude, and layering of the volcanic cloud, in conjunction with optical information, are available for most parts of Europe derived from the observations by the European Aerosol Research Lidar NETwork (EARLINET). Based on multi-wavelength Raman lidar systems, EARLINET is the only instrument worldwide that is able to provide dense time series of high-quality optical data to be used for aerosol typing and for the retrieval of particle microphysical properties as a function of altitude. In this work we show the four-dimensional (4-D) distribution of the Eyjafjallajökull volcanic cloud in the troposphere over Europe as observed by EARLINET during the entire volcanic event (15 April-26 May 2010). All optical properties directly measured (backscatter, extinction, and particle linear depolarization ratio) are stored in the EARLINET database available at www.earlinet.org. A specific relational database providing the volcanic mask over Europe, realized ad hoc for this specific event, has been developed and is available on request at www.earlinet.org. During the first days after the eruption, volcanic particles were detected over Central Europe within a wide range of altitudes, from the upper troposphere down to the local planetary boundary layer (PBL). After 19 April 2010, volcanic particles were detected over southern and south-eastern Europe. During the first half of May (5-15 May), material emitted by the Eyjafjallajökull volcano was detected over Spain and Portugal and then over the Mediterranean and the Balkans. The last observations of the event were recorded until 25 May in Central Europe and in the Eastern Mediterranean area. The 4-D distribution of volcanic aerosol layering and optical properties on European scale reported here provides an unprecedented data set for evaluating satellite data and aerosol dispersion models for this kind of volcanic events.Peer reviewe

LIVAS: A 3-D multi-wavelength aerosol/cloud database based on CALIPSO and EARLINET

We present LIVAS (LIdar climatology of Vertical Aerosol Structure for space-based lidar simulation studies), a 3-D multi-wavelength global aerosol and cloud optical database, optimized to be used for future space-based lidar end-to-end simulations of realistic atmospheric scenarios as well as retrieval algorithm testing activities. The LIVAS database provides averaged profiles of aerosol optical properties for the potential spaceborne laser operating wavelengths of 355, 532, 1064, 1570 and 2050 nm and of cloud optical properties at the wavelength of 532 nm. The global database is based on CALIPSO observations at 532 and 1064 nm and on aerosol-type-dependent backscatter- and extinction-related Ångström exponents, derived from EARLINET (European Aerosol Research Lidar Network) ground-based measurements for the UV and scattering calculations for the IR wavelengths, using a combination of input data from AERONET, suitable aerosol models and recent literature. The required spectral conversions are calculated for each of the CALIPSO aerosol types and are applied to CALIPSO backscatter and extinction data corresponding to the aerosol type retrieved by the CALIPSO aerosol classification scheme. A cloud optical database based on CALIPSO measurements at 532 nm is also provided, neglecting wavelength conversion due to approximately neutral scattering behavior of clouds along the spectral range of LIVAS. Averages of particle linear depolarization ratio profiles at 532 nm are provided as well. Finally, vertical distributions for a set of selected scenes of specific atmospheric phenomena (e.g., dust outbreaks, volcanic eruptions, wild fires, polar stratospheric clouds) are analyzed and spectrally converted so as to be used as case studies for spaceborne lidar performance assessments. The final global data set includes 4-year (1 January 2008–31 December 2011) time-averaged CALIPSO (Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations) data on a uniform grid of 1° × 1° with the original high vertical resolution of CALIPSO in order to ensure realistic simulations of the atmospheric variability in lidar end-to-end simulations

A Dual-Color Fluorescence-Based Platform to Identify Selective Inhibitors of Akt Signaling

Background: Inhibition of Akt signaling is considered one of the most promising therapeutic strategies for many cancers. However, rational target-orientated approaches to cell based drug screens for anti-cancer agents have historically been compromised by the notorious absence of suitable control cells. Methodology/Principal Findings: In order to address this fundamental problem, we have developed BaFiso, a live-cell screening platform to identify specific inhibitors of this pathway. BaFiso relies on the co-culture of isogenic cell lines that have been engineered to sustain interleukin-3 independent survival of the parental Ba/F3 cells, and that are individually tagged with different fluorescent proteins. Whilst in the first of these two lines cell survival in the absence of IL-3 is dependent on the expression of activated Akt, the cells expressing constitutively-activated Stat5 signaling display IL-3 independent growth and survival in an Akt-independent manner. Small molecules can then be screened in these lines to identify inhibitors that rescue IL-3 dependence. Conclusions/Significance: BaFiso measures differential cell survival using multiparametric live cell imaging and permits selective inhibitors of Akt signaling to be identified. BaFiso is a platform technology suitable for the identification of smal

A methodology for investigating dust model performance using synergistic EARLINET/AERONET dust concentration retrievals

Systematic measurements of dust concentration profiles at a continental scale were recently made possible by the development of synergistic retrieval algorithms using combined lidar and sun photometer data and the establishment of robust remote-sensing networks in the framework of Aerosols, Clouds, and Trace gases Research Infra-Structure Network (ACTRIS)/European Aerosol Research Lidar Network (EARLINET). We present a methodology for using these capabilities as a tool for examining the performance of dust transport models. The methodology includes considerations for the selection of a suitable data set and appropriate metrics for the exploration of the results. The approach is demonstrated for four regional dust transport models (BSC-DREAM8b v2, NMMB/BSC-DUST, DREAM-ABOL, DREAM8-NMME-MACC) using dust observations performed at 10 ACTRIS/EARLINET stations. The observations, which include coincident multi-wavelength lidar and sun photometer measurements, were processed with the Lidar-Radiometer Inversion Code (LIRIC) to retrieve aerosol concentration profiles. The methodology proposed here shows advantages when compared to traditional evaluation techniques that utilize separately the available measurements such as separating the contribution of dust from other aerosol types on the lidar profiles and avoiding model assumptions related to the conversion of concentration fields to aerosol extinction values. When compared to LIRIC retrievals, the simulated dust vertical structures were found to be in good agreement for all models with correlation values between 0.5 and 0.7 in the 1-6 km range, where most dust is typically observed. The absolute dust concentration was typically underestimated with mean bias values of -40 to -20 mu g m(-3) at 2 km, the altitude of maximum mean concentration. The reported differences among the models found in this comparison indicate the benefit of the systematic use of the proposed approach in future dust model evaluation studies

Three-dimensional Numerical Modeling and Computational Fluid Dynamics Simulations to Analyze and Improve Oxygen Availability in the AMC Bioartificial Liver

A numerical model to investigate fluid flow and oxygen (O(2)) transport and consumption in the AMC-Bioartificial Liver (AMC-BAL) was developed and applied to two representative micro models of the AMC-BAL with two different gas capillary patterns, each combined with two proposed hepatocyte distributions. Parameter studies were performed on each configuration to gain insight in fluid flow, shear stress distribution and oxygen availability in the AMC-BAL. We assessed the function of the internal oxygenator, the effect of changes in hepatocyte oxygen consumption parameters in time and the effect of the change from an experimental to a clinical setting. In addition, different methodologies were studied to improve cellular oxygen availability, i.e. external oxygenation of culture medium, culture medium flow rate, culture gas oxygen content (pO(2)) and the number of oxygenation capillaries. Standard operating conditions did not adequately provide all hepatocytes in the AMC-BAL with sufficient oxygen to maintain O(2) consumption at minimally 90% of maximal uptake rate. Cellular oxygen availability was optimized by increasing the number of gas capillaries and pO(2) of the oxygenation gas by a factor two. Pressure drop over the AMC-BAL and maximal shear stresses were low and not considered to be harmful. This information can be used to increase cellular efficiency and may ultimately lead to a more productive AMC-BAL

Authenticity and drug resistance in a panel of acute lymphoblastic leukaemia cell lines

Cell lines are important models for drug resistance in acute lymphoblastic leukaemia (ALL), but are often criticised as being unrepresentative of primary disease. There are also doubts regarding the authenticity of many lines. We have characterised a panel of ALL cell lines for growth and drug resistance and compared data with that published for primary patient specimens. In contrast to the convention that cell lines are highly proliferative, those established in our laboratory grow at rates similar to estimates of leukaemic cells in vivo (doubling time 53–442 h). Authenticity was confirmed by genetic fingerprinting, which also demonstrated the potential stability of long-term cultures. In vitro glucocorticoid resistance correlated well with that measured ex vivo, but all lines were significantly more sensitive to vincristine than primary specimens. Sensitivity to methotrexate was inversely correlated to that of glucocorticoids and L-asparaginase, indicating possible reciprocity in resistance mechanisms. A cell line identified as highly methotrexate resistant (IC50 >8000-fold higher than other lines) was derived from a patient receiving escalating doses of the drug, indicating in vivo selection of resistance as a cause of relapse. Many of these lines are suitable as models to study naturally occurring resistance phenotypes in paediatric ALL

Design concepts for the Cherenkov Telescope Array CTA: an advanced facility for ground-based high-energy gamma-ray astronomy

Ground-based gamma-ray astronomy has had a major breakthrough with the impressive results obtained using systems of imaging atmospheric Cherenkov telescopes. Ground-based gamma-ray astronomy has a huge potential in astrophysics, particle physics and cosmology. CTA is an international initiative to build the next generation instrument, with a factor of 5-10 improvement in sensitivity in the 100 GeV-10 TeV range and the extension to energies well below 100 GeV and above 100 TeV. CTA will consist of two arrays (one in the north, one in the south) for full sky coverage and will be operated as open observatory. The design of CTA is based on currently available technology. This document reports on the status and presents the major design concepts of CTA