Model-Free, Regret-Optimal Best Policy Identification in Online CMDPs

This paper considers the best policy identification (BPI) problem in online Constrained Markov Decision Processes (CMDPs). We are interested in algorithms that are model-free, have low regret, and identify an optimal policy with a high probability. Existing model-free algorithms for online CMDPs with sublinear regret and constraint violation do not provide any convergence guarantee to an optimal policy and provide only average performance guarantees when a policy is uniformly sampled at random from all previously used policies. In this paper, we develop a new algorithm, named Pruning-Refinement-Identification (PRI), based on a fundamental structural property of CMDPs proved in Koole(1988); Ross(1989), which we call limited stochasticity. The property says for a CMDP with $N$ constraints, there exists an optimal policy with at most $N$ stochastic decisions. The proposed algorithm first identifies at which step and in which state a stochastic decision has to be taken and then fine-tunes the distributions of these stochastic decisions. PRI achieves trio objectives: (i) PRI is a model-free algorithm; and (ii) it outputs a near-optimal policy with a high probability at the end of learning; and (iii) in the tabular setting, PRI guarantees $\tilde{\mathcal{O}}(\sqrt{K})$ regret and constraint violation, which significantly improves the best existing regret bound $\tilde{\mathcal{O}}(K^{\frac{4}{5}})$ under a model-free algorithm, where $K$ is the total number of episodes

A fast phenotype approach of 3D point clouds of Pinus massoniana seedlings

The phenotyping of Pinus massoniana seedlings is essential for breeding, vegetation protection, resource investigation, and so on. Few reports regarding estimating phenotypic parameters accurately in the seeding stage of Pinus massoniana plants using 3D point clouds exist. In this study, seedlings with heights of approximately 15-30 cm were taken as the research object, and an improved approach was proposed to automatically calculate five key parameters. The key procedure of our proposed method includes point cloud preprocessing, stem and leaf segmentation, and morphological trait extraction steps. In the skeletonization step, the cloud points were sliced in vertical and horizontal directions, gray value clustering was performed, the centroid of the slice was regarded as the skeleton point, and the alternative skeleton point of the main stem was determined by the DAG single source shortest path algorithm. Then, the skeleton points of the canopy in the alternative skeleton point were removed, and the skeleton point of the main stem was obtained. Last, the main stem skeleton point after linear interpolation was restored, while stem and leaf segmentation was achieved. Because of the leaf morphological characteristics of Pinus massoniana, its leaves are large and dense. Even using a high-precision industrial digital readout, it is impossible to obtain a 3D model of Pinus massoniana leaves. In this study, an improved algorithm based on density and projection is proposed to estimate the relevant parameters of Pinus massoniana leaves. Finally, five important phenotypic parameters, namely plant height, stem diameter, main stem length, regional leaf length, and total leaf number, are obtained from the skeleton and the point cloud after separation and reconstruction. The experimental results showed that there was a high correlation between the actual value from manual measurement and the predicted value from the algorithm output. The accuracies of the main stem diameter, main stem length, and leaf length were 93.5%, 95.7%, and 83.8%, respectively, which meet the requirements of real applications

Anticancer activity of a thymidine quinoxaline conjugate is modulated by cytosolic thymidine pathways

Background High levels of thymidine kinase 1 (TK1) and thymidine phosphorylase (TYMP) are key molecular targets by thymidine therapeutics in cancer treatment. The dual roles of TYMP as a tumor growth factor and a key activation enzyme of anticancer metabolites resulted in a mixed outcome in cancer patients. In this study, we investigated the roles of TK1 and TYMP on a thymidine quinoxaline conjugate to evaluate an alternative to circumvent the contradictive role of TYMP. Methods TK1 and TYMP levels in multiple liver cell lines were assessed along with the cytotoxicity of the thymidine conjugate. Cellular accumulation of the thymidine conjugate was determined with organelle-specific dyes. The impacts of TK1 and TYMP were evaluated with siRNA/shRNA suppression and pseudoviral overexpression. Immunohistochemical analysis was performed on both normal and tumor tissues. In vivo study was carried out with a subcutaneous liver tumor model. Results We found that the thymidine conjugate had varied activities in liver cancer cells with different levels of TK1 and TYMP. The conjugate mainly accumulated at endothelial reticulum and was consistent with cytosolic pathways. TK1 was responsible for the cytotoxicity yet high levels of TYMP counteracted such activities. Levels of TYMP and TK1 in the liver tumor tissues were significantly higher than those of normal liver tissues. Induced TK1 overexpression decreased the selectivity of dT-QX due to the concurring cytotoxicity in normal cells. In contrast, shRNA suppression of TYMP significantly enhanced the selective of the conjugate in vitro and reduced the tumor growth in vivo. Conclusions TK1 was responsible for anticancer activity of dT-QX while levels of TYMP counteracted such an activity. The counteraction by TYMP could be overcome with RNA silencing to significantly enhance the dT-QX selectivity in cancer cells

Provably Efficient Model-Free Algorithms for Non-stationary CMDPs

We study model-free reinforcement learning (RL) algorithms in episodic non-stationary constrained Markov Decision Processes (CMDPs), in which an agent aims to maximize the expected cumulative reward subject to a cumulative constraint on the expected utility (cost). In the non-stationary environment, reward, utility functions, and transition kernels can vary arbitrarily over time as long as the cumulative variations do not exceed certain variation budgets. We propose the first model-free, simulator-free RL algorithms with sublinear regret and zero constraint violation for non-stationary CMDPs in both tabular and linear function approximation settings with provable performance guarantees. Our results on regret bound and constraint violation for the tabular case match the corresponding best results for stationary CMDPs when the total budget is known. Additionally, we present a general framework for addressing the well-known challenges associated with analyzing non-stationary CMDPs, without requiring prior knowledge of the variation budget. We apply the approach for both tabular and linear approximation settings

Business Process Text Sketch Automation Generation Using Large Language Model

Business Process Management (BPM) is gaining increasing attention as it has the potential to cut costs while boosting output and quality. Business process document generation is a crucial stage in BPM. However, due to a shortage of datasets, data-driven deep learning techniques struggle to deliver the expected results. We propose an approach to transform Conditional Process Trees (CPTs) into Business Process Text Sketches (BPTSs) using Large Language Models (LLMs). The traditional prompting approach (Few-shot In-Context Learning) tries to get the correct answer in one go, and it can find the pattern of transforming simple CPTs into BPTSs, but for close-domain and CPTs with complex hierarchy, the traditional prompts perform weakly and with low correctness. We suggest using this technique to break down a difficult CPT into a number of basic CPTs and then solve each one in turn, drawing inspiration from the divide-and-conquer strategy. We chose 100 process trees with depths ranging from 2 to 5 at random, as well as CPTs with many nodes, many degrees of selection, and cyclic nesting. Experiments show that our method can achieve a correct rate of 93.42%, which is 45.17% better than traditional prompting methods. Our proposed method provides a solution for business process document generation in the absence of datasets, and secondly, it becomes potentially possible to provide a large number of datasets for the process model extraction (PME) domain.Comment: 10 pages, 7 figure

Polytropic Influence of TRIB3 rs2295490 Genetic Polymorphism on Response to Antihypertensive Agents in Patients With Essential Hypertension

Tribbles homolog 3 (TRIB3) mediating signaling pathways are closely related to blood pressure regulation. Our previous findings suggested a greater benefit on vascular outcomes in patients carrying TRIB3 (251, A > G, rs2295490) G allele with good glucose and blood pressure control. And TRIB3 (rs2295490) AG/GG genotypes were found to reduce primary vascular events in type 2 diabetic patients who received intensive glucose treatment as compared to those receiving standard glucose treatment. However, the effect of TRIB3 genetic variation on antihypertensives was not clear in essential hypertension patients. A total of 368 patients treated with conventional dosage of antihypertensives (6 groups, grouped by atenolol/bisoprolol, celiprolol, doxazosin, azelnidipine/nitrendipine, imidapril, and candesartan/irbesartan) were enrolled in our study. Genetic variations were successfully identified by sanger sequencing. A linear mixed model analysis was performed to evaluate blood pressures among TRIB3 (251, A > G) genotypes and adjusted for baseline age, gender, body mass index, systolic blood pressure (SBP), diastolic blood pressure (DBP), total cholesterol and other biochemical factors appropriately. Our data suggested that TRIB3 (251, A > G) AA genotype carriers showed better antihypertensive effect than the AG/GG genotype carriers [P = 0.014 for DBP and P = 0.042 for mean arterial pressure (MAP)], with a maximal reduction of DBP by 4.2 mmHg and MAP by 3.56 mmHg after azelnidipine or nitrendipine treatment at the 4th week. Similar tendency of DBP-change and MAP-change was found for imidapril (ACEI) treatment, in which marginally significances were achieved (P = 0.073 and 0.075, respectively). Against that, we found that TRIB3 (251, A > G) AG/GG genotype carriers benefited from antihypertensive therapy of ARBs with a larger DBP-change during the period of observation (P = 0.036). Additionally, stratified analysis revealed an obvious difference of the maximal blood pressure change (13 mmHg for the MAP between male and female patients with AA genotype who took ARBs). Although no significant difference in antihypertensive effect between TRIB3 (251, A > G) genotypes in patients treated with α, β-ADRs was observed, we found significant difference in age-, sex-dependent manner related to α, β-ADRs. In conclusion, our data supported that TRIB3 (251, A > G) genetic polymorphism may serve as a useful biomarker in the treatment of hypertension

Characterization of garlic oil/β-cyclodextrin inclusion complexes and application

Garlic oil is a liquid extracted from garlic that has various natural antibacterial and anti-inflammatory properties and is believed to be used to prevent and treat many diseases. However, the main functional components of garlic oil are unstable. Therefore, in this study, encapsulating garlic oil with cyclodextrin using the saturated co-precipitation method can effectively improve its chemical stability and water solubility and reduce its characteristic odor and taste. After preparation, the microcapsules of garlic oil cyclodextrin were characterized, which proved that the encapsulation was successful. Finally, the results showed that the encapsulated garlic oil still had antioxidant ability and slow-release properties. The final addition to plant-based meat gives them a delicious flavor and adds texture and mouthfeel. Provided a new reference for the flavor application of garlic cyclodextrin micro-capsules in plant-based meat patties

Intensive Glucose Control Reduces the Risk Effect of TRIB3, SMARCD3, and ATF6 Genetic Variation on Diabetic Vascular Complications

Type 2 diabetes mellitus is a complex disease. Our previous study revealed that TRIB3 genetic variations were strongly associated with diabetic vascular complications, although TRIB3 regulation pathways remain poorly understood. We used two extreme treatment groups from a 2 × 2 factorial randomized controlled trial to identify a positive association, which was further validated in patients receiving cross treatment to test the effect of genetic polymorphisms among the different treatment groups. A gene-centric score (GS)-weighted model including the three associated genetic variations TRIB3 rs2295490, ATF6 rs12086247, and SMARCD3 rs58125572 was used. The results of the GS model indicated a 46% reduction in the risk of primary vascular complications in patients bearing more than two risk alleles [hazard ratio (HR) 0.54, 95% confidence interval (CI) 0.38–0.76, p < 0.001], following intensive glucose control treatment when compared with patients who received standard glucose control treatment. Furthermore, these patients benefited from active blood pressure-lowering treatment (HR 0.39, 95% CI 0.24–0.64, p < 0.001). However, no significant difference was observed between the two interventions in patients with fewer than two risk alleles (HR 1.09, 95% CI 0.86–1.39, p = 0.47). These results indicate that genetic variants in these three genes may be useful biomarkers for individualized drug therapy in diabetic patients

Ethnopharmacokinetic- and Activity-Guided Isolation of a New Antidepressive Compound from Fructus Aurantii Found in the Traditional Chinese Medicine Chaihu-Shugan-San: A New Approach and Its Application

Aims. We aimed to identify an antidepressive compound found in traditional Chinese medicine (TCM) by a new approach called ethnopharmacokinetic- and activity-guided isolation (EAGI). Methods. The new approach targets an unknown chromatographic peak produced by an absorbed compound found in oral Chaihu-Shugan-San (CSS) taken by patients with depression. Once the compound was isolated from Fructus Aurantii (FA), spectral data was employed to identify the compound. The effects of this compound, FA, and CSS on depressive behaviors were investigated. Results. The identified compound was merazin hydrate (MH) according to the new approach. MH, FA, and CSS significantly reduced immobility time and increased locomotor activity. The effects of MH, FA and CSS were similar to Fluoxetine at high doses. Conclusion. MH, a compound whose antidepressive effect is similar to FA and CSS, was isolated for the first time from FA via targeting its corresponding unknown chromatographic peak, and its antidepressive effect was compared with FA or CSS. These findings highlight the potential for drug R&D and pharmacological research of ∼100,000 TCMs

A Structural Gel Composite Enabled Robust Underwater Mechanosensing Strategy with High Sensitivity

One of the key challenges in developing gel-based electronics is to achieve robust sensing performance, by overcoming the intrinsic weaknesses such as unwanted swelling induced deformation, signal distortion caused by dehydration, large hysteresis in sensing signal, etc. In this work, we proposed a structural gel composite (SGC) approach by encapsulating the conductive hydrogel/MXene with a lipid gel (Lipogel) layer through an in situ polymerization. The hydrophobic Lipogel coating fulfils the SGC with unique anti-swelling property at an aqueous environment and excellent dehydration feature at an open-air, thus leading to long-term ultra-stability (over 90 days) and durability (over 2000 testing cycles) for underwater mechanosensing applications. As a result, the SGC based mechanoreceptor demonstrates a high and stable sensitivity (GF of 14.5). Moreover, several SGC based conceptual sensors with high sensitivity are developed to unveil their profound potentials in underwater monitoring of human motions, waterproof anti-counterfeiting application, and tactile trajectory tracking