ED2: Environment Dynamics Decomposition World Models for Continuous Control

Model-based reinforcement learning (MBRL) achieves significant sample efficiency in practice in comparison to model-free RL, but its performance is often limited by the existence of model prediction error. To reduce the model error, standard MBRL approaches train a single well-designed network to fit the entire environment dynamics, but this wastes rich information on multiple sub-dynamics which can be modeled separately, allowing us to construct the world model more accurately. In this paper, we propose the Environment Dynamics Decomposition (ED2), a novel world model construction framework that models the environment in a decomposing manner. ED2 contains two key components: sub-dynamics discovery (SD2) and dynamics decomposition prediction (D2P). SD2 discovers the sub-dynamics in an environment automatically and then D2P constructs the decomposed world model following the sub-dynamics. ED2 can be easily combined with existing MBRL algorithms and empirical results show that ED2 significantly reduces the model error, increases the sample efficiency, and achieves higher asymptotic performance when combined with the state-of-the-art MBRL algorithms on various continuous control tasks. Our code is open source and available at https://github.com/ED2-source-code/ED2.Comment: 10 pages, 13 figure

Disaster cassification net: A disaster classification algorithm on remote sensing imagery

As we all know, natural disasters have a great impact on people’s lives and properties, and it is very necessary to deal with disaster categories in a timely and effective manner. In light of this, we propose using tandem stitching to create a new Disaster Cassification network D-Net (Disaster Cassification Net) using the D-Conv, D-Linear, D-model, and D-Layer modules. During the experiment, we compared the proposed method with “CNN” and “Transformer”, we found that disaster cassification net compared to CNN algorithm Params decreased by 26–608 times, FLOPs decreased by up to 21 times, Precision increased by 1.6%–43.5%; we found that disaster cassification net compared to Transformer algorithm Params decreased by 23–149 times, FLOPs decreased by 1.7–10 times, Precision increased by 3.9%–25.9%. Precision increased by 3.9%–25.9%. And found that disaster cassification net achieves the effect of SOTA(State-Of-The-Art) on the disaster dataset; After that, we compared the above-mentioned MobileNet_v2 with the best performance on the classification dataset and CCT network are compared with disaster cassification net on fashion_mnist and CIFAR_100 public datasets, respectively, and the results show that disaster cassification net can still achieve the state-of-the-art classification effect. Therefore, our proposed algorithm can be applied not only to disaster tasks, but also to other classification tasks

PreRoutGNN for Timing Prediction with Order Preserving Partition: Global Circuit Pre-training, Local Delay Learning and Attentional Cell Modeling

Pre-routing timing prediction has been recently studied for evaluating the quality of a candidate cell placement in chip design. It involves directly estimating the timing metrics for both pin-level (slack, slew) and edge-level (net delay, cell delay), without time-consuming routing. However, it often suffers from signal decay and error accumulation due to the long timing paths in large-scale industrial circuits. To address these challenges, we propose a two-stage approach. First, we propose global circuit training to pre-train a graph auto-encoder that learns the global graph embedding from circuit netlist. Second, we use a novel node updating scheme for message passing on GCN, following the topological sorting sequence of the learned graph embedding and circuit graph. This scheme residually models the local time delay between two adjacent pins in the updating sequence, and extracts the lookup table information inside each cell via a new attention mechanism. To handle large-scale circuits efficiently, we introduce an order preserving partition scheme that reduces memory consumption while maintaining the topological dependencies. Experiments on 21 real world circuits achieve a new SOTA R2 of 0.93 for slack prediction, which is significantly surpasses 0.59 by previous SOTA method. Code will be available at: https://github.com/Thinklab-SJTU/EDA-AI.Comment: 13 pages, 5 figures, The 38th Annual AAAI Conference on Artificial Intelligence (AAAI 2024

Learning to Select Cuts for Efficient Mixed-Integer Programming

Cutting plane methods play a significant role in modern solvers for tackling mixed-integer programming (MIP) problems. Proper selection of cuts would remove infeasible solutions in the early stage, thus largely reducing the computational burden without hurting the solution accuracy. However, the major cut selection approaches heavily rely on heuristics, which strongly depend on the specific problem at hand and thus limit their generalization capability. In this paper, we propose a data-driven and generalizable cut selection approach, named Cut Ranking, in the settings of multiple instance learning. To measure the quality of the candidate cuts, a scoring function, which takes the instance-specific cut features as inputs, is trained and applied in cut ranking and selection. In order to evaluate our method, we conduct extensive experiments on both synthetic datasets and real-world datasets. Compared with commonly used heuristics for cut selection, the learning-based policy has shown to be more effective, and is capable of generalizing over multiple problems with different properties. Cut Ranking has been deployed in an industrial solver for large-scale MIPs. In the online A/B testing of the product planning problems with more than $10^7$ variables and constraints daily, Cut Ranking has achieved the average speedup ratio of 12.42% over the production solver without any accuracy loss of solution.Comment: Paper accepted at Pattern Recognition journa

AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Model

Aligning agent behaviors with diverse human preferences remains a challenging problem in reinforcement learning (RL), owing to the inherent abstractness and mutability of human preferences. To address these issues, we propose AlignDiff, a novel framework that leverages RL from Human Feedback (RLHF) to quantify human preferences, covering abstractness, and utilizes them to guide diffusion planning for zero-shot behavior customizing, covering mutability. AlignDiff can accurately match user-customized behaviors and efficiently switch from one to another. To build the framework, we first establish the multi-perspective human feedback datasets, which contain comparisons for the attributes of diverse behaviors, and then train an attribute strength model to predict quantified relative strengths. After relabeling behavioral datasets with relative strengths, we proceed to train an attribute-conditioned diffusion model, which serves as a planner with the attribute strength model as a director for preference aligning at the inference phase. We evaluate AlignDiff on various locomotion tasks and demonstrate its superior performance on preference matching, switching, and covering compared to other baselines. Its capability of completing unseen downstream tasks under human instructions also showcases the promising potential for human-AI collaboration. More visualization videos are released on https://aligndiff.github.io/

Mechanism of Qingchang Suppository on repairing the intestinal mucosal barrier in ulcerative colitis

Ulcerative colitis (UC) is a refractory inflammatory bowel disease, and the outcomes of conventional therapies of UC, including 5-aminosalicylic acid, glucocorticoids, immunosuppressants, and biological agents, are not satisfied with patients and physicians with regard to adverse reactions and financial burden. The abnormality of the intestinal mucosal barrier in the pathogenesis of UC was verified. Qingchang Suppository (QCS) is an herbal preparation and is effective in treating ulcerative proctitis. The mechanism of QCS and its active ingredients have not been concluded especially in mucosal healing. This review elucidated the potential mechanism of QCS from the intestinal mucosal barrier perspective to help exploring future QCS research directions

CLEC7A regulates M2 macrophages to suppress the immune microenvironment and implies poorer prognosis of glioma

BackgroundGliomas constitute a category of malignant tumors originating from brain tissue, representing the majority of intracranial malignancies. Previous research has demonstrated the pivotal role of CLEC7A in the progression of various cancers, yet its specific implications within gliomas remain elusive. The primary objective of this study was to investigate the prognostic significance and immune therapeutic potential of CLEC7A in gliomas through the integration of bioinformatics and clinical pathological analyses.MethodsThis investigation involved examining and validating the relationship between CLEC7A and glioma using samples from Hospital, along with data from TCGA, GEO, GTEx, and CGGA datasets. Subsequently, we explored its prognostic value, biological functions, expression location, and impact on immune cells within gliomas. Finally, we investigated its potential impact on the chemotaxis and polarization of macrophages.ResultsThe expression of CLEC7A is upregulated in gliomas, and its levels escalate with the malignancy of tumors, establishing it as an independent prognostic factor. Functional enrichment analysis revealed a significant correlation between CLEC7A and immune function. Subsequent examination of immune cell differential expression demonstrated a robust association between CLEC7A and M2 macrophages. This conclusion was further substantiated through single-cell analysis, immunofluorescence, and correlation studies. Finally, the knockout of CLEC7A in M2 macrophages resulted in a noteworthy reduction in macrophage chemotaxis and polarization factors.ConclusionCLEC7A expression is intricately linked to the pathology and molecular characteristics of gliomas, establishing its role as an independent prognostic factor for gliomas and influencing macrophage function. It could be a promising target for immunotherapy in gliomas

3D sweater garment style generation based on 3D anthropometric characteristic parameters

This paper proposes a method for three-dimensional style modeling of loose sweaters. Through the correlation analysis of the three-dimensional human body and the classic sweater style, a style model was built on the three-dimensional human body model to realize efficient personalized sweater design and production. First of all, the design model was extracted from the human body model based on the characteristics of the ring-cutting algorithm. Secondly, the loose model of the sweater was established based on the chest, waist, and hip data of the human body. Subsequently, the feature line between the size information and style features was created, and curve interpolation values were combined with joint smoothing methods to generate a multi-faceted sweater style model. Finally, the mapping function was used to flatten the style model, the related styles were woven by operating the computer, and the suitability of the established sweater fabric was verified. The comparison results showed that the accuracy of the style construction of this model was improved. Through the analysis of experimental data, it can be proven that the method proposed in this paper can quickly and accurately establish a three-dimensional style model of a sweater, without the need for repeated measurements to make templates, thus saving development time

Research on Lightweight Disaster Classification Based on High-Resolution Remote Sensing Images

With the increasing frequency of natural disasters becoming, it is very important to classify and identify disasters. We propose a lightweight disaster classification model, which has lower computation and parameter quantities and a higher accuracy than other classification models. For this purpose, this paper specially proposes the SDS-Network algorithm, which is optimized on ResNet, to deal with the above problems of remote sensing images. First, it implements the spatial attention mechanism to improve the accuracy of the algorithm; then, the depth separable convolution is introduced to reduce the number of model calculations and parameters while ensuring the accuracy of the algorithm; finally, the effect of the model is increased by adjusting some hyperparameters. The experimental results show that, compared with the classic AlexNet, ResNet18, VGG16, VGG19, and Densenet121 classification models, the SDS-Network algorithm in this paper has a higher accuracy, and when compared with the lightweight models mobilenet series, shufflenet series, squeezenet series, and mnasnet series, it has lower model complexity and a higher accuracy rate. According to a comprehensive performance comparison of the charts made in this article, it is found that the SDS-Network algorithm is still better than the regnet series algorithm. Furthermore, after verification with a public data set, the SDS-Network algorithm in this paper is found to have a good generalization ability. Thus, we can conclude that the SDS-Network classification model of the algorithm in this paper has a good classification effect, and it is suitable for disaster classification tasks. Finally, it is verified on public data sets that the proposed SDS-Network has good generalization ability and portability