AI Illustrator: Translating Raw Descriptions into Images by Prompt-based Cross-Modal Generation

AI illustrator aims to automatically design visually appealing images for books to provoke rich thoughts and emotions. To achieve this goal, we propose a framework for translating raw descriptions with complex semantics into semantically corresponding images. The main challenge lies in the complexity of the semantics of raw descriptions, which may be hard to be visualized (e.g., "gloomy" or "Asian"). It usually poses challenges for existing methods to handle such descriptions. To address this issue, we propose a Prompt-based Cross-Modal Generation Framework (PCM-Frame) to leverage two powerful pre-trained models, including CLIP and StyleGAN. Our framework consists of two components: a projection module from Text Embeddings to Image Embeddings based on prompts, and an adapted image generation module built on StyleGAN which takes Image Embeddings as inputs and is trained by combined semantic consistency losses. To bridge the gap between realistic images and illustration designs, we further adopt a stylization model as post-processing in our framework for better visual effects. Benefiting from the pre-trained models, our method can handle complex descriptions and does not require external paired data for training. Furthermore, we have built a benchmark that consists of 200 raw descriptions. We conduct a user study to demonstrate our superiority over the competing methods with complicated texts. We release our code at https://github.com/researchmm/AI_Illustrator

Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution

Diffusion models, as a kind of powerful generative model, have given impressive results on image super-resolution (SR) tasks. However, due to the randomness introduced in the reverse process of diffusion models, the performances of diffusion-based SR models are fluctuating at every time of sampling, especially for samplers with few resampled steps. This inherent randomness of diffusion models results in ineffectiveness and instability, making it challenging for users to guarantee the quality of SR results. However, our work takes this randomness as an opportunity: fully analyzing and leveraging it leads to the construction of an effective plug-and-play sampling method that owns the potential to benefit a series of diffusion-based SR methods. More in detail, we propose to steadily sample high-quality SR images from pretrained diffusion-based SR models by solving diffusion ordinary differential equations (diffusion ODEs) with optimal boundary conditions (BCs) and analyze the characteristics between the choices of BCs and their corresponding SR results. Our analysis shows the route to obtain an approximately optimal BC via an efficient exploration in the whole space. The quality of SR results sampled by the proposed method with fewer steps outperforms the quality of results sampled by current methods with randomness from the same pretrained diffusion-based SR model, which means that our sampling method "boosts" current diffusion-based SR models without any additional training

Diffusion Enhancement for Cloud Removal in Ultra-Resolution Remote Sensing Imagery

The presence of cloud layers severely compromises the quality and effectiveness of optical remote sensing (RS) images. However, existing deep-learning (DL)-based Cloud Removal (CR) techniques encounter difficulties in accurately reconstructing the original visual authenticity and detailed semantic content of the images. To tackle this challenge, this work proposes to encompass enhancements at the data and methodology fronts. On the data side, an ultra-resolution benchmark named CUHK Cloud Removal (CUHK-CR) of 0.5m spatial resolution is established. This benchmark incorporates rich detailed textures and diverse cloud coverage, serving as a robust foundation for designing and assessing CR models. From the methodology perspective, a novel diffusion-based framework for CR called Diffusion Enhancement (DE) is proposed to perform progressive texture detail recovery, which mitigates the training difficulty with improved inference accuracy. Additionally, a Weight Allocation (WA) network is developed to dynamically adjust the weights for feature fusion, thereby further improving performance, particularly in the context of ultra-resolution image generation. Furthermore, a coarse-to-fine training strategy is applied to effectively expedite training convergence while reducing the computational complexity required to handle ultra-resolution images. Extensive experiments on the newly established CUHK-CR and existing datasets such as RICE confirm that the proposed DE framework outperforms existing DL-based methods in terms of both perceptual quality and signal fidelity

MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation

We propose the first joint audio-video generation framework that brings engaging watching and listening experiences simultaneously, towards high-quality realistic videos. To generate joint audio-video pairs, we propose a novel Multi-Modal Diffusion model (i.e., MM-Diffusion), with two-coupled denoising autoencoders. In contrast to existing single-modal diffusion models, MM-Diffusion consists of a sequential multi-modal U-Net for a joint denoising process by design. Two subnets for audio and video learn to gradually generate aligned audio-video pairs from Gaussian noises. To ensure semantic consistency across modalities, we propose a novel random-shift based attention block bridging over the two subnets, which enables efficient cross-modal alignment, and thus reinforces the audio-video fidelity for each other. Extensive experiments show superior results in unconditional audio-video generation, and zero-shot conditional tasks (e.g., video-to-audio). In particular, we achieve the best FVD and FAD on Landscape and AIST++ dancing datasets. Turing tests of 10k votes further demonstrate dominant preferences for our model. The code and pre-trained models can be downloaded at https://github.com/researchmm/MM-Diffusion.Comment: Accepted by CVPR 202

P4HA2 hydroxylates SUFU to regulate the paracrine Hedgehog signaling and promote B-cell lymphoma progression.

Aberrations in the Hedgehog (Hh) signaling pathway are significantly prevailed in various cancers, including B-cell lymphoma. A critical facet of Hh signal transduction involves the dynamic regulation of the suppressor of fused homolog (SUFU)-glioma-associated oncogene homolog (GLI) complex within the kinesin family member 7 (KIF7)-supported ciliary tip compartment. However, the specific post-translational modifications of SUFU-GLI complex within this context have remained largely unexplored. Our study reveals a novel regulatory mechanism involving prolyl 4-hydroxylase 2 (P4HA2), which forms a complex with KIF7 and is essential for signal transduction of Hh pathway. We demonstrate that, upon Hh pathway activation, P4HA2 relocates alongside KIF7 to the ciliary tip. Here, it hydroxylates SUFU to inhibit its function, thus amplifying the Hh signaling. Moreover, the absence of P4HA2 significantly impedes B lymphoma progression. This effect can be attributed to the suppression of Hh signaling in stromal fibroblasts, resulting in decreased growth factors essential for malignant proliferation of B lymphoma cells. Our findings highlight the role of P4HA2-mediated hydroxylation in modulating Hh signaling and propose a novel stromal-targeted therapeutic strategy for B-cell lymphoma

Optimization of photovoltaic panel deployment in centralized photovoltaic power plant under multiple factors

Solar energy is one of the main renewable energy sources and has rapidly developed in many countries. However, the photovoltaic (PV) output power will be different under various meteorological and geographical conditions. Therefore, this paper presents an optimization method for the deployment of PV panels in a centralized PV power plant considering multiple factors. Firstly, the whole planning area is divided into a certain amount of sub-areas according to a given area, and fuzzy C-means algorithm is used for terrain clustering according to the geographical characteristics of the sub-areas. Secondly, the correlation analysis between each meteorological factor and PV output power is carried out separately to select the main factors affecting PV output power, and then the expected annual PV output power under the joint action of several main meteorological factors in each terrain is calculated by dual-stage attention mechanism based long short-term memory algorithm. Finally, according to the expected annual PV output of each terrain, considering the constraints including cost, area and so on, the deployment optimization of PV panels is obtained to maximize the annual PV output of the whole PV power plant and minimize the construction cost. The results of case studies show that the proposed methods effectively improve the expected PV output power of the PV power plant and reduce the construction cost

Association between prothrombin time-international normalized ratio and prognosis of post-cardiac arrest patients: A retrospective cohort study

BackgroundCardiac arrest (CA) can activate blood coagulation. This study aimed to explore the potential prognostic value of prothrombin time–international normalized ratio (INR) in post-CA patients.MethodsThe clinical data of eligible subjects diagnosed with CA was extracted from the MIMIC-IV database as the training cohort. Restricted cubic spline (RCS), Kaplan–Meier (K-M) survival curve, and Cox regression analyses were conducted to elucidate the association between the INR and all-cause mortality of post-CA patients. Subgroup analysis, propensity score matching (PSM), and inverse probability of treatment (IPTW) were also conducted to improve stability and reliability. Data of the validation cohort were collected from the eICU database, and logistic-regression analyses were performed to verify the findings of the training cohort.ResultsA total of 1,324 subjects were included in the training cohort. A linear correlation existed between INR and the risk of all-cause death of post-CA patients, as shown in RCS analysis, with a hazard ratio (HR) >1 when INR exceeded 1.2. K-M survival curve preliminarily indicated that subjects with INR ≥ 1.2 presented lower survival rate and shorter survival time, and the high level of INR was independently associated with 30-day, 90-day, 1-year, and in-hospital mortalities, with multivariate-adjusted HR of 1.44 (1.20, 1.73), 1.46 (1.23, 1.74), 1.44 (1.23, 1.69), and 1.37 (1.14, 1.64), respectively. These findings were consistent and robust across the subgroup analysis, PSM and IPTW analyses, and validation cohort.ConclusionsWe systematically and comprehensively demonstrated that elevated INR was associated with increased short- and long-term all-cause mortality of post-CA patients. Therefore, elevated INR may be a promising biomarker with prognosis significance

A randomized, double-blind, positive-controlled, Phase-II clinical trial to evaluate efficacy and safety of Fuke Qianjin capsule in Pakistani patients with pelvic inflammatory disease

Ethnopharmacological relevance: Pelvic inflammatory disease (PID) is a frequently occurring gynecological disorder mainly caused by the inflammation of a woman’s upper genital tract. Generally, antibiotics are used for treating PID, but prolonged use poses potential risks of gut bacterial imbalance, bacterial resistance, super bacteria production, and associated adverse reactions. Traditional Chinese medicine (TCM) has shown unique advantages in various ailments and has received widespread clinical research attention. Fuke Qianjin (FUKE) capsule is an approved National Medical Products Administration (NMPA License No. Z20020024) Chinese herbal prescription that has been widely used individually or in combination with other Western medicines for the treatment of various gynecological inflammatory diseases, including chronic cervicitis, endometritis, and chronic PID.Aim: This clinical trial was designed to assess the safety and efficacy of FUKE capsule in mild-to-moderate symptomatic PID patients.Materials and methods: This phase 2, randomized, double-blind, positive controlled clinical trial was conducted in mild-to-moderate symptomatic PID patients at a single center in Pakistan from 21 September 2021 to 11 March 2022. Eligible female participants were randomly assigned to a test and a control group with a ratio of 1:1. The test group subjects received two metronidazole (METRO) tablets and one doxycycline hyclate (DOXY) simulant at a time, twice daily for 14 days, and two Fuke Qianjin (FUKE) capsules, three times a day after a meal for 28 days. Subjects in the control group received two METRO tablets and one DOXY tablet at a time, twice daily for 14 days, and two FUKE simulant capsules, three times a day after meal for 28 days. The primary efficacy outcome was an improvement in pelvic pain symptoms assessed through a visual analog scale (VAS). The secondary outcomes were the improvement in secondary efficacy symptoms like local physical signs, clinical assessment of leucorrhea and cervical secretions through laboratory examination, and improvement in the maximum area of pelvic effusion assessed through gynecological ultrasound after the treatment. The safety outcomes were assessed through vital signs, laboratory tests, electrocardiogram findings, and adverse events/serious adverse events.Results: A total of 198 subjects with active PID were randomly assigned to a test group (n = 99) and a control group (n = 99). The baseline characteristics of the subjects in the two groups were similar. In the intention-to-treat analysis, the primary efficacy was 84.9% for the test group and 71.6% for the control group, with a statistically significant difference (p = 0.0370; 95% CI −0.2568 to −0.0088). The secondary clinical efficacy was 88.4% for the test group and 82.7% for the control group, with no significant difference (p = 0.2977; 95% CI −0.1632 to 0.0501). The improvement in local physical signs was 95.8% for the test group and 76.9% for the control group, with no significant difference (p = 0.0542; 95% CI −0.3697 to −0.0085). The inter-group non-inferiority comparison showed that the upper limit of the 95% CI was less than 0.15 and thus met the non-inferiority requirements of the test group to the control group. The results of clinical signs of leucorrhea and cervical secretions showed that there was no difference in the rate of improvement between the test and control groups, indicating that FUKE was non-inferior to DOXY. A total of 14 adverse events in eight subjects were observed in the trial, with an incidence rate of 4.7%. Four subjects in each group experienced seven adverse events with 4.5% and 4.8% incidence rates of adverse reactions in the test and control groups, with no statistically significant differences (p = 0.2001). No serious adverse events occurred in the trial.Conclusion: The results of this trial indicate that the test drug (Fuke Qianjin capsule) is non-inferior to the control drug (doxycycline hyclate tablet) in treating mild-to-moderate PID patients with comparable efficacy, safety, and tolerability to the control drug.Clinical Trial Registration:www.clinicaltrials.gov, identifier NCT04723069

A teosinte-derived allele of ZmSC improves salt tolerance in maize

Maize, a salt-sensitive crop, frequently suffers severe yield losses due to soil salinization. Enhancing salt tolerance in maize is crucial for maintaining yield stability. To address this, we developed an introgression line (IL76) through introgressive hybridization between maize wild relatives Zea perennis, Tripsacum dactyloides, and inbred Zheng58, utilizing the tri-species hybrid MTP as a genetic bridge. Previously, genetic variation analysis identified a polymorphic marker on Zm00001eb244520 (designated as ZmSC), which encodes a vesicle-sorting protein described as a salt-tolerant protein in the NCBI database. To characterize the identified polymorphic marker, we employed gene cloning and homologous cloning techniques. Gene cloning analysis revealed a non-synonymous mutation at the 1847th base of ZmSCIL76, where a guanine-to-cytosine substitution resulted in the mutation of serine to threonine at the 119th amino acid sequence (using ZmSCZ58 as the reference sequence). Moreover, homologous cloning demonstrated that the variation site derived from Z. perennis. Functional analyses showed that transgenic Arabidopsis lines overexpressing ZmSCZ58 exhibited significant reductions in leaf number, root length, and pod number, alongside suppression of the expression of genes in the SOS and CDPK pathways associated with Ca2+ signaling. Similarly, fission yeast strains expressing ZmSCZ58 displayed inhibited growth. In contrast, the ZmSCIL76 allele from Z. perennis alleviated these negative effects in both Arabidopsis and yeast, with the lines overexpressing ZmSCIL76 exhibiting significantly higher abscisic acid (ABA) content compared to those overexpressing ZmSCZ58. Our findings suggest that ZmSC negatively regulates salt tolerance in maize by suppressing downstream gene expression associated with Ca2+ signaling in the CDPK and SOS pathways. The ZmSCIL76 allele from Z. perennis, however, can mitigate this negative regulatory effect. These results provide valuable insights and genetic resources for future maize salt tolerance breeding programs