Learning a Multi-Agent Controller for Shared Energy Storage System

Deployment of shared energy storage systems (SESS) allows users to use the stored energy to meet their own energy demands while saving energy costs without installing private energy storage equipment. In this paper, we consider a group of building users in the community with SESS, and each user can schedule power injection from the grid as well as SESS according to their demand and real-time electricity price to minimize energy cost and meet energy demand simultaneously. SESS is encouraged to charge when the price is low, thus providing as much energy as possible for users while achieving energy savings. However, due to the complex dynamics of buildings and real-time external signals, it is a challenging task to find high-performance power dispatch decisions in real-time. By designing a multi-agent reinforcement learning framework with state-aware reward functions, SESS and users can realize power scheduling to meet the users' energy demand and SESS's charging/discharging balance without additional communication, so as to achieve energy optimization. Compared with the baseline approach without the participation of the SESS, the energy cost is saved by around 2.37% to 21.58%.Comment: Accepted to 2023 IEEE PES General Meetin

Laxity-Aware Scalable Reinforcement Learning for HVAC Control

Demand flexibility plays a vital role in maintaining grid balance, reducing peak demand, and saving customers' energy bills. Given their highly shiftable load and significant contribution to a building's energy consumption, Heating, Ventilation, and Air Conditioning (HVAC) systems can provide valuable demand flexibility to the power systems by adjusting their energy consumption in response to electricity price and power system needs. To exploit this flexibility in both operation time and power, it is imperative to accurately model and aggregate the load flexibility of a large population of HVAC systems as well as designing effective control algorithms. In this paper, we tackle the curse of dimensionality issue in modeling and control by utilizing the concept of laxity to quantify the emergency level of each HVAC operation request. We further propose a two-level approach to address energy optimization for a large population of HVAC systems. The lower level involves an aggregator to aggregate HVAC load laxity information and use least-laxity-first (LLF) rule to allocate real-time power for individual HVAC systems based on the controller's total power. Due to the complex and uncertain nature of HVAC systems, we leverage a reinforcement learning (RL)-based controller to schedule the total power based on the aggregated laxity information and electricity price. We evaluate the temperature control and energy cost saving performance of a large-scale group of HVAC systems in both single-zone and multi-zone scenarios, under varying climate and electricity market conditions. The experiment results indicate that proposed approach outperforms the centralized methods in the majority of test scenarios, and performs comparably to model-based method in some scenarios.Comment: In Submissio

Sp1, Instead of AhR, Regulates the Basal Transcription of Porcine CYP1A1 at the Proximal Promoter

Pigs are commonly used as an animal model to evaluate the toxic effects of exogenous compounds. Cytochrome P450 1A1 (CYP1A1) metabolizes numerous exogenous compounds and is abundantly expressed in the liver, kidneys, and intestines. The high amino acid similarity between human and porcine CYP1A1 indicates that they probably have the same metabolic characteristics. Therefore, understanding the regulatory mechanism of CYP1A1 expression in pigs is particularly important for predicting the toxicology and metabolic kinetics of exogenous chemicals. Currently, the transcriptional regulation of porcine CYP1A1 has rarely been studied, especially regarding basal transcription. In this study, we first confirmed that the key regulatory elements of porcine CYP1A1 basal transactivation are in the proximal promoter region using promoter truncation analysis via a dual luciferase assay in a porcine kidney cell line LLC-PK1. Two overlapping cis-elements, the xenobiotic response element (XRE) and GC box, in this proximal region potentially play key roles in the basal transactivation of porcine CYP1A1. Furthermore, using electrophoretic mobility shift assay and chromatin immunoprecipitation, the GC box binding protein Sp1 was confirmed to bind to the proximal promoter of porcine CYP1A1, instead of AhR, the XRE binding protein. In LLC-PK1 cells, by knocking down either Sp1 or AhR, the expression of porcine CYP1A1 at the mRNA level and protein level was significantly downregulated, suggesting both proteins are important for porcine CYP1A1 expression. However, promoter activity analysis in LLC-PK1 cells treated with an AhR agonist and antagonist confirmed that AhR does not participate in the basal regulation of porcine CYP1A1 at the proximal promoter. In conclusion, our study revealed that the proximal promoter is the key regulatory region for porcine CYP1A1 basal expression. Although AhR plays an important role in the transactivation of porcine CYP1A1 expression, the key determinant transcription factor for its basal transactivation is Sp1 at the proximal promoter of porcine CYP1A1

Causal inference methods for combining randomized trials and observational studies: a review

With increasing data availability, causal treatment effects can be evaluated across different datasets, both randomized controlled trials (RCTs) and observational studies. RCTs isolate the effect of the treatment from that of unwanted (confounding) co-occurring effects. But they may struggle with inclusion biases, and thus lack external validity. On the other hand, large observational samples are often more representative of the target population but can conflate confounding effects with the treatment of interest. In this paper, we review the growing literature on methods for causal inference on combined RCTs and observational studies, striving for the best of both worlds. We first discuss identification and estimation methods that improve generalizability of RCTs using the representativeness of observational data. Classical estimators include weighting, difference between conditional outcome models, and doubly robust estimators. We then discuss methods that combine RCTs and observational data to improve (conditional) average treatment effect estimation, handling possible unmeasured confounding in the observational data. We also connect and contrast works developed in both the potential outcomes framework and the structural causal model framework. Finally, we compare the main methods using a simulation study and real world data to analyze the effect of tranexamic acid on the mortality rate in major trauma patients. Code to implement many of the methods is provided

A Self-enhancement Approach for Domain-specific Chatbot Training via Knowledge Mining and Digest

Large Language Models (LLMs), despite their great power in language generation, often encounter challenges when dealing with intricate and knowledge-demanding queries in specific domains. This paper introduces a novel approach to enhance LLMs by effectively extracting the relevant knowledge from domain-specific textual sources, and the adaptive training of a chatbot with domain-specific inquiries. Our two-step approach starts from training a knowledge miner, namely LLMiner, which autonomously extracts Question-Answer pairs from relevant documents through a chain-of-thought reasoning process. Subsequently, we blend the mined QA pairs with a conversational dataset to fine-tune the LLM as a chatbot, thereby enriching its domain-specific expertise and conversational capabilities. We also developed a new evaluation benchmark which comprises four domain-specific text corpora and associated human-crafted QA pairs for testing. Our model shows remarkable performance improvement over generally aligned LLM and surpasses domain-adapted models directly fine-tuned on domain corpus. In particular, LLMiner achieves this with minimal human intervention, requiring only 600 seed instances, thereby providing a pathway towards self-improvement of LLMs through model-synthesized training data.Comment: Work in progres

The tale of TILs in breast cancer : a report from the International Immuno-Oncology Biomarker Working Group

The advent of immune-checkpoint inhibitors (ICI) in modern oncology has significantly improved survival in several cancer settings. A subgroup of women with breast cancer (BC) has immunogenic infiltration of lymphocytes with expression of programmed deathligand 1 (PD-L1). These patients may potentially benefit from ICI targeting the programmed death 1 (PD-1)/PD-L1 signaling axis. The use of tumor-infiltrating lymphocytes (TILs) as predictive and prognostic biomarkers has been under intense examination. Emerging data suggest that TILs are associated with response to both cytotoxic treatments and immunotherapy, particularly for patients with triple-negative BC. In this review from The International Immuno-Oncology Biomarker Working Group, we discuss (a) the biological understanding of TILs, (b) their analytical and clinical validity and efforts toward the clinical utility in BC, and (c) the current status of PD-L1 and TIL testing across different continents, including experiences from low-to-middle-income countries, incorporating also the view of a patient advocate. This information will help set the stage for future approaches to optimize the understanding and clinical utilization of TIL analysis in patients with BC.The National Health and Medical Research Council of Australia; the Cure; the Royal Australasian College of Physicians; the NIH/NCI ; the National Breast Cancer Foundation of Australia Endowed Chair; the Breast Cancer Research Foundation, New York and the Breast Cancer Research Foundation (BCRF).www.nature.com/npjbcanceram2022Immunolog

Report on computational assessment of Tumor Infiltrating Lymphocytes from the International Immuno-Oncology Biomarker Working Group

Funder: U.S. Department of Health & Human Services | NIH | National Cancer Institute (NCI)Funder: National Center for Research Resources under award number 1 C06 RR12463-01, VA Merit Review Award IBX004121A from the United States Department of Veterans Affairs Biomedical Laboratory Research and Development Service, the DOD Prostate Cancer Idea Development Award (W81XWH-15-1-0558), the DOD Lung Cancer Investigator-Initiated Translational Research Award (W81XWH-18-1-0440), the DOD Peer Reviewed Cancer Research Program (W81XWH-16-1-0329), the Ohio Third Frontier Technology Validation Fund, the Wallace H. Coulter Foundation Program in the Department of Biomedical Engineering and the Clinical and Translational Science Award Program (CTSA) at Case Western Reserve University.Funder: Susan G Komen Foundation (CCR CCR18547966) and a Young Investigator Grant from the Breast Cancer Alliance.Funder: The Canadian Cancer SocietyFunder: Breast Cancer Research Foundation (BCRF), Grant No. 17-194Abstract: Assessment of tumor-infiltrating lymphocytes (TILs) is increasingly recognized as an integral part of the prognostic workflow in triple-negative (TNBC) and HER2-positive breast cancer, as well as many other solid tumors. This recognition has come about thanks to standardized visual reporting guidelines, which helped to reduce inter-reader variability. Now, there are ripe opportunities to employ computational methods that extract spatio-morphologic predictive features, enabling computer-aided diagnostics. We detail the benefits of computational TILs assessment, the readiness of TILs scoring for computational assessment, and outline considerations for overcoming key barriers to clinical translation in this arena. Specifically, we discuss: 1. ensuring computational workflows closely capture visual guidelines and standards; 2. challenges and thoughts standards for assessment of algorithms including training, preanalytical, analytical, and clinical validation; 3. perspectives on how to realize the potential of machine learning models and to overcome the perceptual and practical limits of visual scoring

Pitfalls in assessing stromal tumor infiltrating lymphocytes (sTILs) in breast cancer

Abstract: Stromal tumor-infiltrating lymphocytes (sTILs) are important prognostic and predictive biomarkers in triple-negative (TNBC) and HER2-positive breast cancer. Incorporating sTILs into clinical practice necessitates reproducible assessment. Previously developed standardized scoring guidelines have been widely embraced by the clinical and research communities. We evaluated sources of variability in sTIL assessment by pathologists in three previous sTIL ring studies. We identify common challenges and evaluate impact of discrepancies on outcome estimates in early TNBC using a newly-developed prognostic tool. Discordant sTIL assessment is driven by heterogeneity in lymphocyte distribution. Additional factors include: technical slide-related issues; scoring outside the tumor boundary; tumors with minimal assessable stroma; including lymphocytes associated with other structures; and including other inflammatory cells. Small variations in sTIL assessment modestly alter risk estimation in early TNBC but have the potential to affect treatment selection if cutpoints are employed. Scoring and averaging multiple areas, as well as use of reference images, improve consistency of sTIL evaluation. Moreover, to assist in avoiding the pitfalls identified in this analysis, we developed an educational resource available at www.tilsinbreastcancer.org/pitfalls

Application of a risk-management framework for integration of stromal tumor-infiltrating lymphocytes in clinical trials

Funder: Breast Cancer Research Foundation (BCRF); doi: https://doi.org/10.13039/100001006Abstract: Stromal tumor-infiltrating lymphocytes (sTILs) are a potential predictive biomarker for immunotherapy response in metastatic triple-negative breast cancer (TNBC). To incorporate sTILs into clinical trials and diagnostics, reliable assessment is essential. In this review, we propose a new concept, namely the implementation of a risk-management framework that enables the use of sTILs as a stratification factor in clinical trials. We present the design of a biomarker risk-mitigation workflow that can be applied to any biomarker incorporation in clinical trials. We demonstrate the implementation of this concept using sTILs as an integral biomarker in a single-center phase II immunotherapy trial for metastatic TNBC (TONIC trial, NCT02499367), using this workflow to mitigate risks of suboptimal inclusion of sTILs in this specific trial. In this review, we demonstrate that a web-based scoring platform can mitigate potential risk factors when including sTILs in clinical trials, and we argue that this framework can be applied for any future biomarker-driven clinical trial setting