Large AI Models in Health Informatics: Applications, Challenges, and the Future

Large AI models, or foundation models, are models recently emerging with massive scales both parameter-wise and data-wise, the magnitudes of which can reach beyond billions. Once pretrained, large AI models demonstrate impressive performance in various downstream tasks. A prime example is ChatGPT, whose capability has compelled people's imagination about the far-reaching influence that large AI models can have and their potential to transform different domains of our lives. In health informatics, the advent of large AI models has brought new paradigms for the design of methodologies. The scale of multi-modal data in the biomedical and health domain has been ever-expanding especially since the community embraced the era of deep learning, which provides the ground to develop, validate, and advance large AI models for breakthroughs in health-related areas. This article presents a comprehensive review of large AI models, from background to their applications. We identify seven key sectors in which large AI models are applicable and might have substantial influence, including 1) bioinformatics; 2) medical diagnosis; 3) medical imaging; 4) medical informatics; 5) medical education; 6) public health; and 7) medical robotics. We examine their challenges, followed by a critical discussion about potential future directions and pitfalls of large AI models in transforming the field of health informatics.Comment: This article has been accepted for publication in IEEE Journal of Biomedical and Health Informatic

Baichuan 2: Open Large-scale Language Models

Large language models (LLMs) have demonstrated remarkable performance on a variety of natural language tasks based on just a few examples of natural language instructions, reducing the need for extensive feature engineering. However, most powerful LLMs are closed-source or limited in their capability for languages other than English. In this technical report, we present Baichuan 2, a series of large-scale multilingual language models containing 7 billion and 13 billion parameters, trained from scratch, on 2.6 trillion tokens. Baichuan 2 matches or outperforms other open-source models of similar size on public benchmarks like MMLU, CMMLU, GSM8K, and HumanEval. Furthermore, Baichuan 2 excels in vertical domains such as medicine and law. We will release all pre-training model checkpoints to benefit the research community in better understanding the training dynamics of Baichuan 2.Comment: Baichuan 2 technical report. Github: https://github.com/baichuan-inc/Baichuan

Simulation research on ONAN transformer winding temperature field based on temperature rise test

Studying the oil-natural-air-natural (ONAN) transformer’s temperature field distribution characteristics and hot spot temperature rise during operation is a key step to evaluate the thermal insulation life of this type of transformer. Firstly, considering the transverse oil passages between the windings and the guiding effect of the oil baffle on the oil flow, this paper takes 35 kV ONAN transformer windings as the research object, which establishes the corresponding electromagnetic-thermal-fluid coupling model and uses the finite element method to calculate the overall distribution of the internal winding temperature field. The calculation results show that when the ONAN transformer reaches thermal equilibrium, the axial temperature distribution of the windings is extremely uneven due to the influence of the transformer oil flow rate, whose difference between both ends is as high as 17.4℃. However, the existence of the oil baffle increases the flow velocity of the inter-turn transverse oil passage to 12 times of the original. Affected by this, the oil temperature difference between the two ends of the oil baffle is as high as 2.42℃. Secondly, this paper conducted a short-circuit temperature rise test on the test prototype, and selected the different axial heights (18%, 72%, 96%, etc.) of the windings on both sides of the transformer as the characteristic sample points for embedding temperature measurement fiber for real-time temperature monitoring. After comparison, it is found that the simulation calculation is basically consistent with experiment data, and the relative error of the two is less than 3.3%

Surface modification of fly ash spheroidal particles and their application in the adsorption of phosphorus and chromium(VI) from single and competitive solute systems

This work focuses on the surface modification of fly ash spheroidal particles and their application in phosphorus and chromium(VI) adsorption. The results show that through surface modification, amorphous silica-alumina gels precipitated on the spheroidal particle surface (by which the microsurface area of the reaction products is effectively enlarged) and the surface zeta potential was changed to fit for adsorbing anions. During the adsorption experiment (single and competitive solute systems), chromium(VI) was easier to adsorb. The surface zeta potential and the existence of competitive ions should be recognized as two important factors affecting adsorption efficiency. A higher temperature could improve the adsorption efficiencies of the two solute systems. The fitting results of the pseudo-second-order model (single and competitive solute systems) show better agreement than those of the pseudo-first-order model at every temperature. The Langmuir adsorption isotherm equation can better simulate the adsorption process in single solute sy039stems, but only the chromium(VI) adsorption process can be fitted by the competitive Langmuir adsorption isotherm in competitive solute systems

Phenolic metabolites as therapeutic in inflammation and neoplasms: Molecular pathways explaining their efficacy

Polyphenols, also known as phenolic compounds, are chemical substances containing aromatic rings as well as at least two hydroxyl groups. Natural phenolic compounds exist widely in plants, which protect plants from ultraviolet radiation and other insults. Phenolic compounds have superior pharmacological and nutritional properties (antimicrobial, antibacterial, antiviral, anti-sclerosis, antioxidant, and anti-inflammatory activities), which have been paid more and more attention by the scientific community. Phenols can protect key cellular components from reactive free radical damage, which is mainly due to their property to activate antioxidant enzymes and alleviate oxidative stress and inflammation. It can also inhibit or isolate reactive oxygen species and transfer electrons to free radicals, thereby avoiding cell damage. It has a regulatory role in glucose metabolism, which has a promising prospect in the prevention and intervention of diabetes. It also prevents cardiovascular disease by regulating blood pressure and blood lipids. Polyphenols can inhibit cell proliferation by affecting Erk1/2, CDK, and PI3K/Akt signaling pathways. Polyphenols can function as enhancers of intrinsic defense systems, including superoxide dismutase (SOD) and glutathione peroxidase (GPX). Simultaneously, they can modulate multiple proteins and transcription factors, making them promising candidates in the investigation of anti-cancer medications. This review focuses on multiple aspects of phenolic substances, including their natural origins, production process, disinfection activity, oxidative and anti-inflammatory functions, and the effects of different phenolic substances on tumors

Detection of Advanced Glycosylation End Products in the Cornea Based on Molecular Fluorescence and Machine Learning

Advanced glycosylation end products (AGEs) are continuously produced and accumulated in the bodies of diabetic patients. To effectively predict disease trends in diabetic patients, a corneal fluorescence detection device was designed based on the autofluorescence properties of AGEs, and corneal fluorescence measurements were performed on 83 volunteers. Multiple linear regression (MLR), extreme gradient boosting (XGBoost), support vector regression (SVR), and back-propagation neural network (BPNN) were used to predict the human AGE content. Physiological parameters which may affect corneal AGE content were collected for a correlation analysis to select the features that had a strong correlation with the corneal concentration of AGEs to participate in modeling. By comparing the predictive effects of the four models in the two cases of a single-input feature and a multi-input feature, it was found that the model with the single-input feature had a better predictive effect. In this case, corneal AGE content was predicted by a single-input SVR model, with the average error rate (AER), mean square error (MSE), and determination coefficient R-squared (R2) of the SVR model calculated as 2.43%, 0.026, and 0.932, respectively. These results proved the potential of our method and device for noninvasive detection of the concentration of AGEs in the cornea

Factors Influencing Pharmaceutical and Personal Care Product Degradation in Aqueous Solution Using Pulsed Wave Ultrasound

Continuous wave (CW) and pulsed wave (PW) ultrasound were used to degrade pharmaceuticals (carbamazepine, ibuprofen, acetaminophen, sulfamethoxazole, and ciprofloxacin), and personal care products (propyl gallate and diethyl phthalate). These compounds, covering a range of physicochemical properties and a diversity of structures, were explored to determine if and how PW ultrasound is advantageous to CW ultrasound. Degradation rates by PW ultrasound were faster for smaller compounds and slower for larger compounds than that under CW ultrasound. The addition of a bulk solution ^•OH trapping agent, acetic acid, to PPCP solutions indicates that the fraction of degradation occurring in bulk solution is positively correlated with the molar volume of the compound. Overall, smaller PPCP compounds with molar volumes less than 130 mL/mol are able to more readily diffuse to bubble interfaces and are impacted most by pulsing ultrasound

AMOM: Adaptive Masking over Masking for Conditional Masked Language Model

Transformer-based autoregressive (AR) methods have achieved appealing performance for varied sequence-to-sequence generation tasks, e.g., neural machine translation, summarization, and code generation, but suffer from low inference efficiency. To speed up the inference stage, many non-autoregressive (NAR) strategies have been proposed in the past few years. Among them, the conditional masked language model (CMLM) is one of the most versatile frameworks, as it can support many different sequence generation scenarios and achieve very competitive performance on these tasks. In this paper, we further introduce a simple yet effective adaptive masking over masking strategy to enhance the refinement capability of the decoder and make the encoder optimization easier. Experiments on 3 different tasks (neural machine translation, summarization, and code generation) with 15 datasets in total confirm that our proposed simple method achieves significant performance improvement over the strong CMLM model. Surprisingly, our proposed model yields state-of-the-art performance on neural machine translation (34.62 BLEU on WMT16 EN to RO, 34.82 BLEU on WMT16 RO to EN, and 34.84 BLEU on IWSLT De to En) and even better performance than the AR Transformer on 7 benchmark datasets with at least 2.2x speedup. Our code is available at GitHub