Quasi-optimal Learning with Continuous Treatments

Many real-world applications of reinforcement learning (RL) require making decisions in continuous action environments. In particular, determining the optimal dose level plays a vital role in developing medical treatment regimes. One challenge in adapting existing RL algorithms to medical applications, however, is that the popular infinite support stochastic policies, e.g., Gaussian policy, may assign riskily high dosages and harm patients seriously. Hence, it is important to induce a policy class whose support only contains near-optimal actions, and shrink the action-searching area for effectiveness and reliability. To achieve this, we develop a novel \emph{quasi-optimal learning algorithm}, which can be easily optimized in off-policy settings with guaranteed convergence under general function approximations. Theoretically, we analyze the consistency, sample complexity, adaptability, and convergence of the proposed algorithm. We evaluate our algorithm with comprehensive simulated experiments and a dose suggestion real application to Ohio Type 1 diabetes dataset.Comment: The first two authors contributed equally to this wor

Policy Learning for Individualized Treatment Regimes on Infinite Time Horizon

With the recent advancements of technology in facilitating real-time monitoring and data collection, "just-in-time" interventions can be delivered via mobile devices to achieve both real-time and long-term management and control. Reinforcement learning formalizes such mobile interventions as a sequence of decision rules and assigns treatment arms based on the user's status at each decision point. In practice, real applications concern a large number of decision points beyond the time horizon of the currently collected data. This usually refers to reinforcement learning in the infinite horizon setting, which becomes much more challenging. This article provides a selective overview of some statistical methodologies on this topic. We discuss their modeling framework, generalizability, and interpretability and provide some use case examples. Some future research directions are discussed in the end

AI in Pharma for Personalized Sequential Decision-Making: Methods, Applications and Opportunities

In the pharmaceutical industry, the use of artificial intelligence (AI) has seen consistent growth over the past decade. This rise is attributed to major advancements in statistical machine learning methodologies, computational capabilities and the increased availability of large datasets. AI techniques are applied throughout different stages of drug development, ranging from drug discovery to post-marketing benefit-risk assessment. Kolluri et al. provided a review of several case studies that span these stages, featuring key applications such as protein structure prediction, success probability estimation, subgroup identification, and AI-assisted clinical trial monitoring. From a regulatory standpoint, there was a notable uptick in submissions incorporating AI components in 2021. The most prevalent therapeutic areas leveraging AI were oncology (27%), psychiatry (15%), gastroenterology (12%), and neurology (11%). The paradigm of personalized or precision medicine has gained significant traction in recent research, partly due to advancements in AI techniques \cite{hamburg2010path}. This shift has had a transformative impact on the pharmaceutical industry. Departing from the traditional "one-size-fits-all" model, personalized medicine incorporates various individual factors, such as environmental conditions, lifestyle choices, and health histories, to formulate customized treatment plans. By utilizing sophisticated machine learning algorithms, clinicians and researchers are better equipped to make informed decisions in areas such as disease prevention, diagnosis, and treatment selection, thereby optimizing health outcomes for each individual

Mimicking the Catalytic Center for the Water-Splitting Reaction in Photosystem II

The oxygen-evolving center (OEC) in photosystem II (PSII) of plants, algae and cyanobacteria is a unique natural catalyst that splits water into electrons, protons and dioxygen. The crystallographic studies of PSII have revealed that the OEC is an asymmetric Mn4CaO5-cluster. The understanding of the structure-function relationship of this natural Mn4CaO5-cluster is impeded mainly due to the complexity of the protein environment and lack of a rational chemical model as a reference. Although it has been a great challenge for chemists to synthesize the OEC in the laboratory, significant advances have been achieved recently. Different artificial complexes have been reported, especially a series of artificial Mn4CaO4-clusters that closely mimic both the geometric and electronic structures of the OEC in PSII, which provides a structurally well-defined chemical model to investigate the structure-function relationship of the natural Mn4CaO5-cluster. The deep investigations on this artificial Mn4CaO4-cluster could provide new insights into the mechanism of the water-splitting reaction in natural photosynthesis and may help the development of efficient catalysts for the water-splitting reaction in artificial photosynthesis

A Calibration-Free Hybrid BCI Speller System Based on High-Frequency SSVEP and sEMG

Hybrid brain-computer interface (hBCI) systems that combine steady-state visual evoked potential (SSVEP) and surface electromyography (sEMG) signals have attracted attention of researchers due to the advantage of exhibiting significantly improved system performance. However, almost all existing studies adopt low-frequency SSVEP to build hBCI. It produces much more visual fatigue than high-frequency SSVEP. Therefore, the current study attempts to build a hBCI based on high-frequency SSVEP and sEMG. With these two signals, this study designed and realized a 32-target hBCI speller system. Thirty-two targets were separated from the middle into two groups. Each side contained 16 sets of targets with different high-frequency visual stimuli (i.e., 31-34.75 Hz with an interval of 0.25 Hz). sEMG was utilized to choose the group and SSVEP was adopted to identify intra-group targets. The filter bank canonical correlation analysis (FBCCA) and the root mean square value (RMS) methods were used to identify signals. Therefore, the proposed system allowed users to operate it without system calibration. A total of 12 healthy subjects participated in online experiment, with an average accuracy of 93.52 ± 1.66% and the average information transfer rate (ITR) reached 93.50 ± 3.10 bits/min. Furthermore, 12 participants perfectly completed the free-spelling tasks. These results of the experiments indicated feasibility and practicality of the proposed hybrid BCI speller system

Recent research progress on the correlation between metabolic syndrome and Helicobacter pylori infection

Background Globally, metabolic syndrome (MS) and Helicobacter pylori (HP) infection, which have gained an epidemic status, are major challenges to human health, society, and medical professionals. Recent studies have demonstrated that MS is closely related to HP infection. Additionally, HP is an important risk factor for gastric cancer. However, systematic reviews on HP are lacking. This review aimed to summarize and analyze the potential correlation of HP infection with MS and its components, as well as the underlying mechanism, to provide reference and strategies for clinical prevention and treatment. Methodology Previous studies examining the correlation between HP and MS since 1990 were retrieved from the PubMed, Web of Science, and Embase databases. The potential correlation between HP infection and MS and its components was comprehensively analyzed. The keywords “Helicobacter pylori,” “HP,” “metabolic syndrome,” “hypertension,” “obesity,” “diabetes,” or “dyslipidemia” were used in all fields. No language restrictions were imposed. Results MS was strongly correlated to HP infection. The inflammatory response and inflammatory factors produced during HP infection are important etiological factors for insulin resistance and MS. The co-occurrence of long-term chronic inflammation and immune dysfunction with MS may be the predisposing factor for HP infection. MS components, such as diabetes, hypertension, dyslipidemia, and obesity were also correlated with HP infection in one or both directions. Conclusions HP infection and MS may promote the pathogenesis of each other. The contribution of HP infection and MS to gastric cancer cannot be ruled out based on co-occurrence. The MS components diabetes and obesity may be bidirectionally correlated with HP infection

Highly efficient isolation and 3D printing of fibroblasts for cultured meat production

Fibroblasts are important components of animal tissues such as muscle and skin, as they are the major producers of various matrix proteins. Matrix proteins such as collagen play an important role in meat products by providing unique nutrition, texture, and flavor. Cultured meat is an innovative meat alternative produced by culturing animal cells, but currently, relatively few studies have been conducted using fibroblasts as seed cells for cultured meat manufacturing. In this work, we first developed an innovative digestion-friction method for isolating fibroblasts from porcine skin efficiently and cost-effectively. After optimizing the enzymatic digestion and physical friction conditions, 2.39 ± 0.28 × 105 fibroblasts were obtained from 1 cm2 of porcine skin tissue, which was about 9 times higher than the conventional tissue explant method. In addition, we identified an edible bio-ink composed of gelatin and chitosan that has good printing properties and supports fibroblast adhesion and growth. Furthermore, we fabricated fibroblast-based cultured meat by 3D printing with an initial cell density of 1.0 × 107 mL−1 and evaluated its texture and nutritional properties. This work provides valuable insights and references for introducing fibroblasts into the production of cultured meat that is more comparable to structured animal meat