Cross-Modal Concept Learning and Inference for Vision-Language Models

Large-scale pre-trained Vision-Language Models (VLMs), such as CLIP, establish the correlation between texts and images, achieving remarkable success on various downstream tasks with fine-tuning. In existing fine-tuning methods, the class-specific text description is matched against the whole image. We recognize that this whole image matching is not effective since images from the same class often contain a set of different semantic objects, and an object further consists of a set of semantic parts or concepts. Individual semantic parts or concepts may appear in image samples from different classes. To address this issue, in this paper, we develop a new method called cross-model concept learning and inference (CCLI). Using the powerful text-image correlation capability of CLIP, our method automatically learns a large set of distinctive visual concepts from images using a set of semantic text concepts. Based on these visual concepts, we construct a discriminative representation of images and learn a concept inference network to perform downstream image classification tasks, such as few-shot learning and domain generalization. Extensive experimental results demonstrate that our CCLI method is able to improve the performance upon the current state-of-the-art methods by large margins, for example, by up to 8.0% improvement on few-shot learning and by up to 1.3% for domain generalization

BDC-Adapter: Brownian Distance Covariance for Better Vision-Language Reasoning

Large-scale pre-trained Vision-Language Models (VLMs), such as CLIP and ALIGN, have introduced a new paradigm for learning transferable visual representations. Recently, there has been a surge of interest among researchers in developing lightweight fine-tuning techniques to adapt these models to downstream visual tasks. We recognize that current state-of-the-art fine-tuning methods, such as Tip-Adapter, simply consider the covariance between the query image feature and features of support few-shot training samples, which only captures linear relations and potentially instigates a deceptive perception of independence. To address this issue, in this work, we innovatively introduce Brownian Distance Covariance (BDC) to the field of vision-language reasoning. The BDC metric can model all possible relations, providing a robust metric for measuring feature dependence. Based on this, we present a novel method called BDC-Adapter, which integrates BDC prototype similarity reasoning and multi-modal reasoning network prediction to perform classification tasks. Our extensive experimental results show that the proposed BDC-Adapter can freely handle non-linear relations and fully characterize independence, outperforming the current state-of-the-art methods by large margins.Comment: Accepted by BMVC 202

Platelet Distribution Width Levels Can Be a Predictor in the Diagnosis of Persistent Organ Failure in Acute Pancreatitis

Purpose. The change of serum platelet indices such as platelet distribution width (PDW) has been reported in a series of inflammatory reaction and clinical diseases. However, the relationship between PDW and the incidence of persistent organ failure (POF) in acute pancreatitis (AP) has not been elucidated so far. Materials and Methods. A total of 135 patients with AP admitted within 72 hours from symptom onset of AP at our center between December 2014 and January 2016 were included in this retrospective study. Demographic parameters on admission, organ failure assessment, laboratory data, and in-hospital mortality were compared between patients with and without POF. Multivariable logistic regression analyses were utilized to evaluate the predictive value of serum PDW for POF. Results. 30 patients were diagnosed with POF. Compared to patients without POF, patients with POF showed a significantly higher value of serum PDW on admission (14.88 ± 2.24 versus 17.60 ± 1.96%, P<0.001). After multivariable analysis, high PDW level remained a risk factor for POF (odds ratio 39.42, 95% CI: 8.64–179.77; P<0.001). A PDW value of 16.45% predicted POF with an area under the curve (AUC) of 0.870, a sensitivity with 0.867, and a specificity with 0.771, respectively. Conclusions. Our results indicate that serum PDW on admission could be a predictive factor in AP with POF and may serve as a potential prognostic factor

Self-Correctable and Adaptable Inference for Generalizable Human Pose Estimation

A central challenge in human pose estimation, as well as in many other machine learning and prediction tasks, is the generalization problem. The learned network does not have the capability to characterize the prediction error, generate feedback information from the test sample, and correct the prediction error on the fly for each individual test sample, which results in degraded performance in generalization. In this work, we introduce a self-correctable and adaptable inference (SCAI) method to address the generalization challenge of network prediction and use human pose estimation as an example to demonstrate its effectiveness and performance. We learn a correction network to correct the prediction result conditioned by a fitness feedback error. This feedback error is generated by a learned fitness feedback network which maps the prediction result to the original input domain and compares it against the original input. Interestingly, we find that this self-referential feedback error is highly correlated with the actual prediction error. This strong correlation suggests that we can use this error as feedback to guide the correction process. It can be also used as a loss function to quickly adapt and optimize the correction network during the inference process. Our extensive experimental results on human pose estimation demonstrate that the proposed SCAI method is able to significantly improve the generalization capability and performance of human pose estimation.Comment: Accepted by CVPR 202

Risk factors of pancreatic fistula after resection of pancreatic body and tail duct adenocarcinoma

Objective: To investigate the risk factors of pancreatic fistula after resection of pancreatic body and tail duct adenocarcinoma. Methods all cases of pancreatic body and tail resection for pancreatic body and tail duct adenocarcinoma in Union Hospital Affiliated to Tongji Medical College of Huazhong University of science and technology from January 2016 to December 2018 were analyzed retrospectively in a single center. The preoperative, intraoperative and postoperative data were collected and analyzed by spssv22.0. The definition and grouping of pancreatic fistula were implemented according to the standards formulated by the international pancreatic fistula research group in 2016. All cases were followed up for at least 3 months. Results: A a total of 91 cases were included in the study. The overall pancreatic fistula rate was 25.27% (23/91). No death occurred within 90 days after operation. Three risk factors for pancreatic fistula were identified: Pancreatic texture (soft) [odds ratio =8.965,95% confidence interval (2.400,33.490), p=0.001], combined with cardiovascular disease [odds ratio =9.148,95% confidence interval (1.936,43.225), p=0.05], albumin &lt;26.50g/l[odds ratio =6.100,95% confidence interval (1.846,20.157), p=0.003]. Conclusion soft pancreas, complicated with cardiovascular disease and low albumin level on the first day after operation are independent risk factors for pancreatic fistula after operation of pancreatic duct adenocarcinoma. Due to the limitations of the study, the results need to be further verified

Communication Efficiency Optimization of Federated Learning for Computing and Network Convergence of 6G Networks

Federated learning effectively addresses issues such as data privacy by collaborating across participating devices to train global models. However, factors such as network topology and device computing power can affect its training or communication process in complex network environments. A new network architecture and paradigm with computing-measurable, perceptible, distributable, dispatchable, and manageable capabilities, computing and network convergence (CNC) of 6G networks can effectively support federated learning training and improve its communication efficiency. By guiding the participating devices' training in federated learning based on business requirements, resource load, network conditions, and arithmetic power of devices, CNC can reach this goal. In this paper, to improve the communication efficiency of federated learning in complex networks, we study the communication efficiency optimization of federated learning for computing and network convergence of 6G networks, methods that gives decisions on its training process for different network conditions and arithmetic power of participating devices in federated learning. The experiments address two architectures that exist for devices in federated learning and arrange devices to participate in training based on arithmetic power while achieving optimization of communication efficiency in the process of transferring model parameters. The results show that the method we proposed can (1) cope well with complex network situations (2) effectively balance the delay distribution of participating devices for local training (3) improve the communication efficiency during the transfer of model parameters (4) improve the resource utilization in the network.Comment: 13 pages, 11 figures, accepted by Frontiers of Information Technology & Electronic Engineerin

Adversarial Attacks on Fairness of Graph Neural Networks

Fairness-aware graph neural networks (GNNs) have gained a surge of attention as they can reduce the bias of predictions on any demographic group (e.g., female) in graph-based applications. Although these methods greatly improve the algorithmic fairness of GNNs, the fairness can be easily corrupted by carefully designed adversarial attacks. In this paper, we investigate the problem of adversarial attacks on fairness of GNNs and propose G-FairAttack, a general framework for attacking various types of fairness-aware GNNs in terms of fairness with an unnoticeable effect on prediction utility. In addition, we propose a fast computation technique to reduce the time complexity of G-FairAttack. The experimental study demonstrates that G-FairAttack successfully corrupts the fairness of different types of GNNs while keeping the attack unnoticeable. Our study on fairness attacks sheds light on potential vulnerabilities in fairness-aware GNNs and guides further research on the robustness of GNNs in terms of fairness. The open-source code is available at https://github.com/zhangbinchi/G-FairAttack.Comment: 32 pages, 5 figure

Rule-Based Automatic Generation of Mediator Patterns for Service Composition Mismatches

To perform service composition, mismatches are challenging obstacles due to the decentralization and independent development of services. Mediation, as a promising solution, attracts most attentions. And pattern based mediation proposed a modularly constructive thoughtway: Basic mediator patterns were created and sufficient for advanced mediators against all possible mismatches. The pattern structure is illustrated in this paper. And construction rules for each pattern are presented. Executable codes such as BPEL codes can be automatically generated from these rules. As a systematic engineering solution, its feasibility is validated through a case study in the end

An Accurate and Efficient Time Delay Estimation Method of Ultra-High Frequency Signals for Partial Discharge Localization in Substations

Partial discharge (PD) localization in substations based on the ultra-high frequency (UHF) method can be used to efficiently assess insulation conditions. Localization accuracy is affected by the accuracy of the time delay (TD) estimation, which is critical for PD localization in substations. A review of existing TD estimation methods indicates that there is a need to develop methods that are both accurate and computationally efficient. In this paper, a novel TD estimation method is proposed to improve both accuracy and efficiency. The TD is calculated using an improved cross-correlation algorithm based on full-wavefronts of array UHF signals, which are extracted using the minimum cumulative energy method and zero-crossing points searching methods. The cross-correlation algorithm effectively suppresses the TD error caused by differences between full-wavefronts. To verify the method, a simulated PD source test in a laboratory and a field test in a 220 kV substation were carried out. The results show that the proposed method is accurate even in case of low signal-to-noise ratio, but with greatly improved computational efficiency