Conservative State Value Estimation for Offline Reinforcement Learning

Offline reinforcement learning faces a significant challenge of value over-estimation due to the distributional drift between the dataset and the current learned policy, leading to learning failure in practice. The common approach is to incorporate a penalty term to reward or value estimation in the Bellman iterations. Meanwhile, to avoid extrapolation on out-of-distribution (OOD) states and actions, existing methods focus on conservative Q-function estimation. In this paper, we propose Conservative State Value Estimation (CSVE), a new approach that learns conservative V-function via directly imposing penalty on OOD states. Compared to prior work, CSVE allows more effective in-data policy optimization with conservative value guarantees. Further, we apply CSVE and develop a practical actor-critic algorithm in which the critic does the conservative value estimation by additionally sampling and penalizing the states \emph{around} the dataset, and the actor applies advantage weighted updates extended with state exploration to improve the policy. We evaluate in classic continual control tasks of D4RL, showing that our method performs better than the conservative Q-function learning methods and is strongly competitive among recent SOTA methods

Solving the Batch Stochastic Bin Packing Problem in Cloud: A Chance-constrained Optimization Approach

This paper investigates a critical resource allocation problem in the first party cloud: scheduling containers to machines. There are tens of services and each service runs a set of homogeneous containers with dynamic resource usage; containers of a service are scheduled daily in a batch fashion. This problem can be naturally formulated as Stochastic Bin Packing Problem (SBPP). However, traditional SBPP research often focuses on cases of empty machines, whose objective, i.e., to minimize the number of used machines, is not well-defined for the more common reality with nonempty machines. This paper aims to close this gap. First, we define a new objective metric, Used Capacity at Confidence (UCaC), which measures the maximum used resources at a probability and is proved to be consistent for both empty and nonempty machines, and reformulate the SBPP under chance constraints. Second, by modeling the container resource usage distribution in a generative approach, we reveal that UCaC can be approximated with Gaussian, which is verified by trace data of real-world applications. Third, we propose an exact solver by solving the equivalent cutting stock variant as well as two heuristics-based solvers -- UCaC best fit, bi-level heuristics. We experimentally evaluate these solvers on both synthetic datasets and real application traces, demonstrating our methodology's advantage over traditional SBPP optimal solver minimizing the number of used machines, with a low rate of resource violations.Comment: To appear in SIGKDD 2022 as Research Track pape

Polystyrene microplastics protect lettuce (Lactuca sativa) from the hazardous effects of Cu(OH)2 nanopesticides

Copper-based nanopesticides are released into the environment during foliar spray application, and they could, on their own or in combination with microplastics (MPs), pose threats to environmental safety and human health. In this study, Cu(OH)2 nanowires greatly decreased the vigor of lettuce seeds (p< 0.01) and the root length of lettuce seedlings (p< 0.01) and significantly altered the lettuce antioxidant defence system and MDA content (p< 0.05). Released Cu2+ played a critical role in the toxicity mechanism of Cu(OH)2 nanowires in lettuce seedlings, as evidenced by the substantial accumulation of Cu in the seedling roots (p< 0.01) rather than in the leaves. Polystyrene (PS) MPs (1 mg/L) stimulated lettuce seedling growth, as shown by the (highly) significant increase in root and leaf length and in the seed vigor index (p< 0.01 or 0.05). Notably, PS MPs (1 mg/L) neutralized the hazardous effects of 1 mg/L Cu(OH)2 nanowire treatment on lettuce growth, as reflected by the vitality and root length of the seedlings returning to normal levels. The PS MPs (1 mg/L) absorbed on middle root surfaces and strongly hindered Cu accumulation in lettuce roots, which was the predominant mechanism by which PS MPs suppressed the hazardous effects of the Cu(OH)2 nanowires. This study strengthens the understanding of the toxicity and toxicity mechanisms of Cu(OH)2 nanowires with or without PS MPs in the environment

The distribution of heterophilic antigens and their relationship with autoimmune diseases

IntroductionMicrobial infections are associated with the occurrence of autoimmune diseases, but the mechanisms of microbial infection inducing autoimmune diseases are not fully understood. The existence of heterophilic antigens between microorganisms and human tissues may explain part of the pathogenesis of autoimmune diseases. Here, we investigate the distribution of heterophilic antigens and its relationship with autoimmune diseases.MethodsMonoclonal antibodies against a variety of microorganisms were prepared. The titer, subclass and reactivity of antibodies with microorganisms were identified, and heterophilic antibodies that cross-reacted with human tissues were screened by human tissue microarray. The reactivity of these heterophilic antibodies with different individuals and different species was further examined by immunohistochemistry.ResultsIn this study, 21 strains of heterophilic antibodies were screened. The results showed that these heterophilic antibodies were produced due to the existence of heterophilic antigens between microorganism and human body and the distribution of heterophilic antigens had individual, tissue and species differences.ConclusionOur study showed that heterophilic antigens exist widely between microorganisms and human body, and the heterophilic antigens carried by microorganisms may break the immune tolerance of the body through carrier effect and initiate immune response, which may be one of the important mechanisms of infection inducing autoimmune diseases

Podwyższone stężenia lipokaliny-2 w surowicy krwi są związane ze wskaźnikami metabolizmu glukozy i metabolizmu kostnego w przebiegu cukrzycy typu 2

Introduction: The role of lipocalin 2 (LCN2) in type 2 diabetes mellitus (T2DM) needs to be fully elucidated. Moreover, bone has been demonstrated to modulate glucose metabolism via LCN2. We thus performed this study to investigate the associations of LCN2 with indexes of glucose metabolism in T2DM. The associations of LCN2 with bone metabolism were examined concurrently. Material and methods: Total 288 Chinese Han subjects entered in this study including 146 patients with T2DM and 142 subjects with normal glucose tolerance. Insulin resistance was assessed by HOMA-IR andWstęp: Wyjaśnienie roli lipokaliny-2 (lipocalin 2; LCN2) w przebiegu cukrzycy typu 2 jest niezbędne, w szczególności, że zostało dowiedzione, iż kość moduluje metabolizm glukozy za pośrednictwem LCN2. Niniejsze badanie przeprowadzono, aby zbadać, w jaki sposób LCN2 jest powiązana ze wskaźnikami metabolizmu glukozy w przebiegu cukrzycy typu 2. Jednocześnie zbadano powiązania LCN2 z metabolizmem kostnym. Materiał i metody: W badaniu wzięło udział 288 Chińczyków Han, w tym 146 pacjentów z cukrzycą typu 2 i 142 pacjentów z prawidłową tolerancją glukozy. Insulinooporność oceniano za pomocą wskaźnika HOMA-IR, natomiast funkcję komórek beta trzustki za pomocą HOMA-β. W przypadku pacjentów z cukrzycą typu 2 oznaczano również markery obrotu kostnego, N-końcowy propeptyd prokolagenu typu I, C-końcowy usieciowany telopeptyd łańcucha alfa kolagenu typu I, gęstość mineralną kości (bone mineral density; BMD) odcinka lędźwiowego kręgosłupa i szyjki kości udowej. Wyniki: Stężenia LCN2 w surowicy krwi w przebiegu cukrzycy typu 2 były wyższe niż u osób z prawidłową tolerancją glukozy (p = 0,005). Ponadto, LCN2 była dodatnio skorelowana ze stężeniem insuliny w surowicy krwi na czczo (r = 0,262, p = 0,001), wskaźnikiem HOMA-IR (r = 0,185, p = 0,026) i HOMA-β (r = 0,306, p < 0,001), odpowiednio, oraz ujemnie skorelowana z osoczowym stężeniem glukozy na czczo (r = –0,218, p = 0,006). Dodatkowo, BMD szyjki kości udowej (β = –0,176, p = 0,033), N-końcowy propeptyd prokolagenu typu I (β = 0,181, p = 0,026) oraz C-końcowy usieciowany telopeptyd łańcucha alfa kolagenu typu I (β = –0,168, p = 0,037) były niezależnymi czynnikami predykcyjnymi dla LCN2 w przebiegu cukrzycy typu 2. Wnioski: Lipokalina-2 była powiązana ze wskaźnikami metabolizmu glukozy. Ponadto, BMD oraz markery obrotu kostnego były nie­zależnymi czynnikami predykcyjnymi dla LCN2 w przebiegu cukrzycy typu 2. Można sądzić, że LCN2 może odgrywać rolę w procesie wzajemnego wpływu homeostazy kości i homeostazy glukozy

The Broad Host Range Phage vB_CpeS_BG3P Is Able to Inhibit Clostridium perfringens Growth

Clostridium perfringens is an important pathogen for both humans and animals, causing human foodborne disease and necrotic enteritis in poultry. In the present study, a C. perfringens-specific phage, vB_CpeS_BG3P (designated as BG3P hereafter), was isolated from chicken farm sewage. Both electron microscopy and phylogenetic analysis suggested that phage BG3P is a novel phage belonging to Siphoviridae family. Phage BG3P exhibited a broad host range against different C. perfringens isolates (90.63% of strains were infected). Sequencing of the complete genome revealed a linear double-stranded DNA (43,528 bp) with 28.65% GC content. After sequence analysis, 73 open reading frames (orf s) were predicted, of which only 13 were annotated with known functions. No tRNA and virulence encoding genes were detected. It should be noted that the protein of orf 15 has 97.92% homology to C. perfringens-specific chloramphenicol resistance protein, which has not been reported for any C. perfringens phage. Phylogenetic analysis of the ssDNA binding protein demonstrated that this phage is closely related to C. perfringens phages phiSM101 and phi3626. In considering future use as an antimicrobial agent, some biological characteristics were observed, such as a good pH (3–11) stability and moderate temperature tolerance (<60 C). Moreover, bacteriophage BG3P showed a good antimicrobial effect against C. perfringens liquid cultures. Thus, phage treatment with MOI ≥ 100 completely inhibited bacterial growth compared to untreated cultures. Although phage BG3P shows good lytic efficiency and broad host range in vitro, future development and application may need to consider removal of the chloramphenicol-like resistance gene or exploring its lysin for future antibacterial applications.This work was supported by the National Key Research and Development Program of China (No. 2018YFE0101900) and the Jiangsu Agricultural Science and Technology Foundation (No. cx(21)1004)

Effect of Internalin InlJ of Listeria monocytogenes on Phage Sensitivity and Biofilm Formation

In order to explore the role and function of the inlJ gene of Listeria monocytogenes (Lm) in phage sensitivity and biofilm, the inlJ gene-deficient strain Lm NJ05-ΔinlJ was constructed by homologous recombination. The growth, adhesion and invasion characteristics of the defective strain were identified. The results showed that compared with the wide-type strain Lm NJ05, the adhesion and invasion of RAW264.7 cells by Lm NJ05-ΔinlJ were significantly reduced to 20.05% and 4.42%, respectively. The efficiency of plaque formation was enhanced by 2.72 folds and phage vB-LmoM-NJ05 had a stronger lytic activity on Lm NJ05-ΔinlJ. Phage vB-LmoM-NJ05 at titers of 105 and 108 PFU/mL could completely inhibit and remove the biofilm of Lm NJ05-ΔinlJ, respectively. Transcriptional analysis of biofilm formation-related genes showed that the transcriptional levels of the degU, agrA, agrD, luxS, yneA, recA and hpt genes were significantly decreased to nearly zero in the defective strain after interacting with phage vB-LmoM-NJ05. In conclusion, deletion of the inlJ gene can enhance the phage sensitivity of Lm, and down-regulate the ability of cell invasion and biofilm formation. Therefore, the inlJ gene not only regulate Lm but also affect its interaction with phage, which lays a foundation for the development and application of phage biocontrol

A small-molecule inhibitor of the aberrant transcription factor CBFβ-SMMHC delays leukemia in mice

This is the author’s version of the work. It is posted here by permission of the AAAS for personal use, not for redistribution. The definitive version was published in Science on 2015 February 13; 347(6223): 779–784, DOI: 10.1126/science.aaa0314.Acute myeloid leukemia (AML) is the most common form of adult leukemia. The transcription factor fusion CBFβ-SMMHC (core binding factor β and the smooth-muscle myosin heavy chain), expressed in AML with the chromosome inversion inv(16)(p13q22), outcompetes wild-type CBFβ for binding to the transcription factor RUNX1, deregulates RUNX1 activity in hematopoiesis, and induces AML. Current inv(16) AML treatment with nonselective cytotoxic chemotherapy results in a good initial response but limited long-term survival. Here, we report the development of a protein-protein interaction inhibitor, AI-10-49, that selectively binds to CBFβ-SMMHC and disrupts its binding to RUNX1. AI-10-49 restores RUNX1 transcriptional activity, displays favorable pharmacokinetics, and delays leukemia progression in mice. Treatment of primary inv(16) AML patient blasts with AI-10-49 triggers selective cell death. These data suggest that direct inhibition of the oncogenic CBFβ-SMMHC fusion protein may be an effective therapeutic approach for inv(16) AML, and they provide support for transcription factor targeted therapy in other cancers

Expert consensus on digital intraoral scanning technology

Digital intraoral scanning is a hot topic in the field of oral digital technology. In recent years, digital intraoral scanning has gradually become the mainstream technology in orthodontics, prosthodontics, and implant dentistry. The precision of digital intraoral scanning and the accuracy and stitching of data collection are the keys to the success of the impression. However, the operators are less familiar with the intraoral scanning characteristics, imaging processing, operator scanning method, oral tissue specificity of the scanned object, and restoration design. Thus far, no unified standard and consensus on digital intraoral scanning technology has been achieved at home or abroad. To deal with the problems encountered in oral scanning and improve the quality of digital scanning, we collected common expert opinions and sought to expound the causes of scanning errors and countermeasures by summarizing the existing evidence. We also describe the scanning strategies under different oral impression requirements. The expert consensus is that due to various factors affecting the accuracy of digital intraoral scanning and the reproducibility of scanned images, adopting the correct scanning trajectory can shorten clinical operation time and improve scanning accuracy. The scanning trajectories mainly include the E-shaped, segmented, and S-shaped methods. When performing fixed denture restoration, it is recommended to first scan the abutment and adjacent teeth. When performing fixed denture restoration, it is recommended to scan the abutment and adjacent teeth first. Then the cavity in the abutment area is excavated. Lastly, the cavity gap was scanned after completing the abutment preparation. This method not only meets clinical needs but also achieves the most reliable accuracy. When performing full denture restoration in edentulous jaws, setting markers on the mucosal tissue at the bottom of the alveolar ridge, simultaneously capturing images of the vestibular area, using different types of scanning paths such as Z-shaped, S-shaped, buccal-palatal and palatal-buccal pathways, segmented scanning of dental arches, and other strategies can reduce scanning errors and improve image stitching and overlap. For implant restoration, when a single crown restoration is supported by implants and a small span upper structure restoration, it is recommended to first pre-scan the required dental arch. Then the cavity in the abutment area is excavated. Lastly, scanning the cavity gap after installing the implant scanning rod. When repairing a bone level implant crown, an improved indirect scanning method can be used. The scanning process includes three steps: First, the temporary restoration, adjacent teeth, and gingival tissue in the mouth are scanned; second, the entire dental arch is scanned after installing a standard scanning rod on the implant; and third, the temporary restoration outside the mouth is scanned to obtain the three-dimensional shape of the gingival contour of the implant neck, thereby increasing the stability of soft tissue scanning around the implant and improving scanning restoration. For dental implant fixed bridge repair with missing teeth, the mobility of the mucosa increases the difficulty of scanning, making it difficult for scanners to distinguish scanning rods of the same shape and size, which can easily cause image stacking errors. Higher accuracy of digital implant impressions can be achieved by changing the geometric shape of the scanning rods to change the optical curvature radius. The consensus confirms that as the range of scanned dental arches and the number of data concatenations increases, the scanning accuracy decreases accordingly, especially when performing full mouth implant restoration impressions. The difficulty of image stitching processing can easily be increased by the presence of unstable and uneven mucosal morphology inside the mouth and the lack of relatively obvious and fixed reference objects, which results in insufficient accuracy. When designing restorations of this type, it is advisable to carefully choose digital intraoral scanning methods to obtain model data. It is not recommended to use digital impressions when there are more than five missing teeth

Clinical Evaluation of Targeted Arterial Infusion of Verapamil in the Interventional Chemotherapy of Primary Hepatocellular Carcinoma

This study evaluates the clinical effectiveness of targeted arterial infusion of verapamil in interventional treatment of primary hepatocellular carcinoma. For this purpose, in 273 patients with middle- or late-stage primary hepatocellular carcinoma, verapamil, IL-2, and chemotherapeutic agents were infused into the target tumor vasculature through femoral artery using Seldinger technique. The medications were infused as serial dilutions, and effectiveness was evaluated after two treatment cycles. Among these 273 patients, 76 cases showed clinical cure or significant improvement, 119 cases improved, 64 cases stabilized, while 14 cases progressed or deteriorated. In 238 patients, KPS score and body weights were stabilized. Regarding side effects, 99 patients (36.3%) developed leukopenia; 160 patients had gastrointestinal reactions (58.6%); 80 patients (29.3%) presented with elevated ALT/AST profile; and 65 cases (23.8%) had pyrexia; however, these side effects abated quickly. No elevations in BUN/Cr and/or allergic reactions were observed. Pre- and post-intervention cardiac function did not change in all the patients. No significant change was observed in ECG. Liver function was also improved after two cycles of treatment. It was concluded that verapamil management via targeted arterial infusion could effectively reverse the multidrug resistance in cancer cells in primary hepatocellular carcinoma patients and therefore enhanced the efficacy of chemotherapy