Safe reinforcement learning with self-improving hard constraints for multi-energy management systems

Safe reinforcement learning (RL) with hard constraint guarantees is a promising optimal control direction for multi-energy management systems. It only requires the environment-specific constraint functions itself a prior and not a complete model (i.e. plant, disturbance and noise models, and prediction models for states not included in the plant model - e.g. demand, weather, and price forecasts). The project-specific upfront and ongoing engineering efforts are therefore still reduced, better representations of the underlying system dynamics can still be learned and modeling bias is kept to a minimum (no model-based objective function). However, even the constraint functions alone are not always trivial to accurately provide in advance (e.g. an energy balance constraint requires the detailed determination of all energy inputs and outputs), leading to potentially unsafe behavior. In this paper, we present two novel advancements: (I) combining the Optlayer and SafeFallback method, named OptLayerPolicy, to increase the initial utility while keeping a high sample efficiency. (II) introducing self-improving hard constraints, to increase the accuracy of the constraint functions as more data becomes available so that better policies can be learned. Both advancements keep the constraint formulation decoupled from the RL formulation, so that new (presumably better) RL algorithms can act as drop-in replacements. We have shown that, in a simulated multi-energy system case study, the initial utility is increased to 92.4% (OptLayerPolicy) compared to 86.1% (OptLayer) and that the policy after training is increased to 104.9% (GreyOptLayerPolicy) compared to 103.4% (OptLayer) - all relative to a vanilla RL benchmark. While introducing surrogate functions into the optimization problem requires special attention, we do conclude that the newly presented GreyOptLayerPolicy method is the most advantageous.Comment: 4579 words. arXiv admin note: text overlap with arXiv:2207.0383

Continuous injection synthesis of indium arsenide quantum dots emissive in the short-wavelength infrared

With the emergence of applications based on short-wavelength infrared light, indium arsenide quantum dots are promising candidates to address existing shortcomings of other infrared-emissive nanomaterials. However, III–V quantum dots have historically struggled to match the high-quality optical properties of II–VI quantum dots. Here we present an extensive investigation of the kinetics that govern indium arsenide nanocrystal growth. Based on these insights, we design a synthesis of large indium arsenide quantum dots with narrow emission linewidths. We further synthesize indium arsenide-based core-shell-shell nanocrystals with quantum yields up to 82% and improved photo- and long-term storage stability. We then demonstrate non-invasive through-skull fluorescence imaging of the brain vasculature of murine models, and show that our probes exhibit 2–3 orders of magnitude higher quantum yields than commonly employed infrared emitters across the entire infrared camera sensitivity range. We anticipate that these probes will not only enable new biomedical imaging applications, but also improved infrared nanocrystal-LEDs and photon-upconversion technology.National Science Foundation (U.S.) (EECS-1449291)National Institutes of Health (U.S.) (Massachusetts Institute of Technology. Laser Biomedical Research Center. 9-P41-EB015871-26A1)Massachusetts Institute of Technology. Institute for Soldier Nanotechnologies (W911NF-13-D-0001)Boehringer Ingelheim FondsEuropean Molecular Biology Organization (Long-term Fellowship)National Science Foundation (U.S.). Graduate Research Fellowship ProgramAmerican Society for Engineering Education. National Defense Science and Engineering Graduate FellowshipUnited States. Dept. of Energy. Center for Excitonics (DE- SC0001088)

Immunopeptidomics toolkit library (IPTK): a python-based modular toolbox for analyzing immunopeptidomics data

Background The human leukocyte antigen (HLA) proteins play a fundamental role in the adaptive immune system as they present peptides to T cells. Mass-spectrometry-based immunopeptidomics is a promising and powerful tool for characterizing the immunopeptidomic landscape of HLA proteins, that is the peptides presented on HLA proteins. Despite the growing interest in the technology, and the recent rise of immunopeptidomics-specific identification pipelines, there is still a gap in data-analysis and software tools that are specialized in analyzing and visualizing immunopeptidomics data. Results We present the IPTK library which is an open-source Python-based library for analyzing, visualizing, comparing, and integrating different omics layers with the identified peptides for an in-depth characterization of the immunopeptidome. Using different datasets, we illustrate the ability of the library to enrich the result of the identified peptidomes. Also, we demonstrate the utility of the library in developing other software and tools by developing an easy-to-use dashboard that can be used for the interactive analysis of the results. Conclusion IPTK provides a modular and extendable framework for analyzing and integrating immunopeptidomes with different omics layers. The library is deployed into PyPI at https://pypi.org/project/IPTKL/ and into Bioconda at https://anaconda.org/bioconda/iptkl , while the source code of the library and the dashboard, along with the online tutorials are available at https://github.com/ikmb/iptoolkit

Safe reinforcement learning for multi-energy management systems with known constraint functions

Reinforcement learning (RL) is a promising optimal control technique for multi-energy management systems. It does not require a model a priori - reducing the upfront and ongoing project-specific engineering effort and is capable of learning better representations of the underlying system dynamics. However, vanilla RL does not provide constraint satisfaction guarantees - resulting in various unsafe interactions within its safety-critical environment. In this paper, we present two novel safe RL methods, namely SafeFallback and GiveSafe, where the safety constraint formulation is decoupled from the RL formulation and which provides hard-constraint satisfaction guarantees both during training (exploration) and exploitation of the (close-to) optimal policy. In a simulated multi-energy systems case study we have shown that both methods start with a significantly higher utility (i.e. useful policy) compared to a vanilla RL benchmark (94,6% and 82,8% compared to 35,5%) and that the proposed SafeFallback method even can outperform the vanilla RL benchmark (102,9% to 100%). We conclude that both methods are viably safety constraint handling techniques capable beyond RL, as demonstrated with random agents while still providing hard-constraint guarantees. Finally, we propose fundamental future work to i.a. improve the constraint functions itself as more data becomes available

Safe reinforcement learning for multi-energy management systems with known constraint functions

Reinforcement learning (RL) is a promising optimal control technique for multi-energy management systems. It does not require a model a priori - reducing the upfront and ongoing project-specific engineering effort and is capable of learning better representations of the underlying system dynamics. However, vanilla RL does not provide constraint satisfaction guarantees — resulting in various potentially unsafe interactions within its environment. In this paper, we present two novel online model-free safe RL methods, namely SafeFallback and GiveSafe, where the safety constraint formulation is decoupled from the RL formulation. These provide hard-constraint satisfaction guarantees both during training and deployment of the (near) optimal policy. This is without the need of solving a mathematical program, resulting in less computational power requirements and more flexible constraint function formulations. In a simulated multi-energy systems case study we have shown that both methods start with a significantly higher utility compared to a vanilla RL benchmark and Optlayer benchmark (94,6% and 82,8% compared to 35,5% and 77,8%) and that the proposed SafeFallback method even can outperform the vanilla RL benchmark (102,9% to 100%). We conclude that both methods are viably safety constraint handling techniques applicable beyond RL, as demonstrated with random policies while still providing hard-constraint guarantees

Establishing a Large-Scale Field Experiment to Assess the Effects of Artificial Light at Night on Species and Food Webs

Artificial light at night (ALAN) is one of the most obvious hallmarks of human presence in an ecosystem. The rapidly increasing use of artificial light has fundamentally transformed nightscapes throughout most of the globe, although little is known about how ALAN impacts the biodiversity and food webs of illuminated ecosystems. We developed a large-scale experimental infrastructure to study the effects of ALAN on a light-naïve, natural riparian (i.e., terrestrial-aquatic) ecosystem. Twelve street lights (20 m apart) arranged in three rows parallel to an agricultural drainage ditch were installed on each of two sites located in a grassland ecosystem in northern Germany. A range of biotic, abiotic, and photometric data are collected regularly to study the short- and long-term effects of ALAN on behavior, species interactions, physiology, and species composition of communities. Here we describe the infrastructure setup and data collection methods, and characterize the study area including photometric measurements. None of the measured parameters differed significantly between sites in the period before illumination. Results of one short-term experiment, carried out with one site illuminated and the other acting as a control, demonstrate the attraction of ALAN by the immense and immediate increase of insect catches at the lit street lights. The experimental setup provides a unique platform for carrying out interdisciplinary research on sustainable lighting

Genetic association study of QT interval highlights role for calcium signaling pathways in myocardial repolarization.

The QT interval, an electrocardiographic measure reflecting myocardial repolarization, is a heritable trait. QT prolongation is a risk factor for ventricular arrhythmias and sudden cardiac death (SCD) and could indicate the presence of the potentially lethal mendelian long-QT syndrome (LQTS). Using a genome-wide association and replication study in up to 100,000 individuals, we identified 35 common variant loci associated with QT interval that collectively explain ∼8-10% of QT-interval variation and highlight the importance of calcium regulation in myocardial repolarization. Rare variant analysis of 6 new QT interval-associated loci in 298 unrelated probands with LQTS identified coding variants not found in controls but of uncertain causality and therefore requiring validation. Several newly identified loci encode proteins that physically interact with other recognized repolarization proteins. Our integration of common variant association, expression and orthogonal protein-protein interaction screens provides new insights into cardiac electrophysiology and identifies new candidate genes for ventricular arrhythmias, LQTS and SCD

Genetic risk and a primary role for cell-mediated immune mechanisms in multiple sclerosis.

Multiple sclerosis is a common disease of the central nervous system in which the interplay between inflammatory and neurodegenerative processes typically results in intermittent neurological disturbance followed by progressive accumulation of disability. Epidemiological studies have shown that genetic factors are primarily responsible for the substantially increased frequency of the disease seen in the relatives of affected individuals, and systematic attempts to identify linkage in multiplex families have confirmed that variation within the major histocompatibility complex (MHC) exerts the greatest individual effect on risk. Modestly powered genome-wide association studies (GWAS) have enabled more than 20 additional risk loci to be identified and have shown that multiple variants exerting modest individual effects have a key role in disease susceptibility. Most of the genetic architecture underlying susceptibility to the disease remains to be defined and is anticipated to require the analysis of sample sizes that are beyond the numbers currently available to individual research groups. In a collaborative GWAS involving 9,772 cases of European descent collected by 23 research groups working in 15 different countries, we have replicated almost all of the previously suggested associations and identified at least a further 29 novel susceptibility loci. Within the MHC we have refined the identity of the HLA-DRB1 risk alleles and confirmed that variation in the HLA-A gene underlies the independent protective effect attributable to the class I region. Immunologically relevant genes are significantly overrepresented among those mapping close to the identified loci and particularly implicate T-helper-cell differentiation in the pathogenesis of multiple sclerosis

Project Report No. 59, Site Index Equations for Loblolly and Slash Pine Plantations in East Texas, Update: Fall 1997

Each published set of equations was developed from analyses of East Texas Pine Plantation Research Project (ETPPRP) data collected from the array of ETPPRP permanent research plots located throughout East Texas

Методы и механизмы геттерирования кремниевых структур в производстве интегральных микросхем

Увеличение степени интеграции элементной базы предъявляет все более жесткие требования к уменьшению концентрации загрязняющих примесей и окислительных дефектов упаковки в исходных кремниевых пластинах с ее сохранением в технологическом цикле изготовления ИМС. Это обуславливает высокую актуальность применения геттерирования в современной технологии микроэлектроники. В статье рассмотрены существующие методы геттерирования кремниевых пластин и механизмы их протекания.Збільшення ступеня інтеграції елементної бази пред'являє все більш жорсткі вимоги до зменшення концентрації забруднюючих домішок та окислювальних дефектів упаковки у вихідних кремнієвих пластинах за її збереження у технологічному циклі виготовлення ІМС. Це обумовлює високу актуальність застосування гетерування в сучасній технології мікроелектроніки. Розглянуто існуючі методи гетерування кремнієвих пластин та розглянуто механізми їх перебігу.Increasing the degree of integration of hardware components imposes more stringent requirements for the reduction of the concentration of contaminants and oxidation stacking faults in the original silicon wafers with its preservation in the IC manufacturing process cycle. This causes high relevance of the application of gettering in modern microelectronic technology. The existing methods of silicon wafers gettering and the mechanisms of their occurrence are considered