Robust and IP-Protecting Vertical Federated Learning against Unexpected Quitting of Parties

Vertical federated learning (VFL) enables a service provider (i.e., active party) who owns labeled features to collaborate with passive parties who possess auxiliary features to improve model performance. Existing VFL approaches, however, have two major vulnerabilities when passive parties unexpectedly quit in the deployment phase of VFL - severe performance degradation and intellectual property (IP) leakage of the active party's labels. In this paper, we propose \textbf{Party-wise Dropout} to improve the VFL model's robustness against the unexpected exit of passive parties and a defense method called \textbf{DIMIP} to protect the active party's IP in the deployment phase. We evaluate our proposed methods on multiple datasets against different inference attacks. The results show that Party-wise Dropout effectively maintains model performance after the passive party quits, and DIMIP successfully disguises label information from the passive party's feature extractor, thereby mitigating IP leakage

Rethinking Normalization Methods in Federated Learning

Federated learning (FL) is a popular distributed learning framework that can reduce privacy risks by not explicitly sharing private data. In this work, we explicitly uncover external covariate shift problem in FL, which is caused by the independent local training processes on different devices. We demonstrate that external covariate shifts will lead to the obliteration of some devices' contributions to the global model. Further, we show that normalization layers are indispensable in FL since their inherited properties can alleviate the problem of obliterating some devices' contributions. However, recent works have shown that batch normalization, which is one of the standard components in many deep neural networks, will incur accuracy drop of the global model in FL. The essential reason for the failure of batch normalization in FL is poorly studied. We unveil that external covariate shift is the key reason why batch normalization is ineffective in FL. We also show that layer normalization is a better choice in FL which can mitigate the external covariate shift and improve the performance of the global model. We conduct experiments on CIFAR10 under non-IID settings. The results demonstrate that models with layer normalization converge fastest and achieve the best or comparable accuracy for three different model architectures.Comment: Submitted to DistributedML'22 worksho

SiDA: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models

Mixture-of-Experts (MoE) has emerged as a favorable architecture in the era of large models due to its inherent advantage, i.e., enlarging model capacity without incurring notable computational overhead. Yet, the realization of such benefits often results in ineffective GPU memory utilization, as large portions of the model parameters remain dormant during inference. Moreover, the memory demands of large models consistently outpace the memory capacity of contemporary GPUs. Addressing this, we introduce SiDA (Sparsity-inspired Data-Aware), an efficient inference approach tailored for large MoE models. SiDA judiciously exploits both the system's main memory, which is now abundant and readily scalable, and GPU memory by capitalizing on the inherent sparsity on expert activation in MoE models. By adopting a data-aware perspective, SiDA achieves enhanced model efficiency with a neglectable performance drop. Specifically, SiDA attains a remarkable speedup in MoE inference with up to 3.93X throughput increasing, up to 75% latency reduction, and up to 80% GPU memory saving with down to 1% performance drop. This work paves the way for scalable and efficient deployment of large MoE models, even in memory-constrained systems

CoRE: A context-aware relation extraction method for relation completion

We identify relation completion (RC) as one recurring problem that is central to the success of novel big data applications such as Entity Reconstruction and Data Enrichment. Given a semantic relation, RC attempts at linking entity pairs between two entity lists under the relation. To accomplish the RC goals, we propose to formulate search queries for each query entity α based on some auxiliary information, so that to detect its target entity β from the set of retrieved documents. For instance, a pattern-based method (PaRE) uses extracted patterns as the auxiliary information in formulating search queries. However, high-quality patterns may decrease the probability of finding suitable target entities. As an alternative, we propose CoRE method that uses context terms learned surrounding the expression of a relation as the auxiliary information in formulating queries. The experimental results based on several real-world web data collections demonstrate that CoRE reaches a much higher accuracy than PaRE for the purpose of RC

IRank: A term-based innovation ranking system for conferences and scholars

Since the proposition of Journal Impact Factor [1] in 1963, the classical citation-based ranking scheme has been a standard criterion to rank journals and conferences. However, the reference of a paper cannot list all relevant publications and the citation relationships are not always available especially when related to copyright problem. Besides, we cannot evaluate a newly published paper before it is cited by others. Therefore, we propose an alternative method, term-based evaluation scheme which can evaluate publications by terms they use. Then we can rank conferences, journals and scholars accordingly. We think this term-based ranking scheme can be used to evaluate innovation quality for conferences and scholars. To evaluate our scheme and to facilitate its application, we develop an innovation ranking system called IRank to rank conferences and authors in the field of Database Systems. The performance of IRank demonstrates the effectiveness of our scheme

Approximate membership localization (AML) for web-based join

In this paper, we propose a search-based approach to join two tables in the absence of clean join attributes. Non-structured documents from the web are used to express the correlations between a given query and a reference list. To implement this approach, a major challenge we meet is how to efficiently determine the number of times and the locations of each clean reference from the reference list that is approximately mentioned in the retrieved documents. We formalize the Approximate Membership Localization (AML) problem and propose an efficient partial pruning algorithm to solve it. A study using real-word data sets demonstrates the effectiveness of our search-based approach, and the efficiency of our AML algorithm

AML: efficient Approximate Membership Localization within a web-based join framework

In this paper, we propose a new type of Dictionary-based Entity Recognition Problem, named Approximate Membership Localization (AML). The popular Approximate Membership Extraction (AME) provides a full coverage to the true matched substrings from a given document, but many redundancies cause a low efficiency of the AME process and deteriorate the performance of real-world applications using the extracted substrings. The AML problem targets at locating nonoverlapped substrings which is a better approximation to the true matched substrings without generating overlapped redundancies. In order to perform AML efficiently, we propose the optimized algorithm P-Prune that prunes a large part of overlapped redundant matched substrings before generating them. Our study using several real-word data sets demonstrates the efficiency of P-Prune over a baseline method. We also study the AML in application to a proposed web-based join framework scenario which is a search-based approach joining two tables using dictionary-based entity recognition from web documents. The results not only prove the advantage of AML over AME, but also demonstrate the effectiveness of our search-based approach

Synthesis, Crystal Structure, Absolute Configuration and Antitumor Activity of the Enantiomers of 5-Bromo-2-chloro-N-(1-phenylethyl)pyridine-3-sulfonamide

Pyridinesulfonamide is an important fragment which has a wide range of applications in novel drugs. R- and S-isomers of 5-bromo-2-chloro-N-(1-phenylethyl)pyridine-3-sulfonamide have been synthesized, and the stereostructures have been researched. Single crystals of both compounds were obtained for X-ray analysis, and the absolute configurations (ACs) have been further confirmed by electronic circular dichroism (ECD), optical rotation (OR) and quantum chemical calculations. The crystal structures and calculated geometries were extremely similar, which permitted a comparison of the relative reliabilities of ACs obtained by ECD analyses and theoretical simulation. In addition, the effect of stereochemistry on the PI3Kα kinase and anticancer activity were investigated. Compounds 10a and 10b inhibit the activity of PI3Kα kinase with IC50 values of 1.08 and 2.69 μM, respectively. Furthermore, molecular docking was performed to analyze the binding modes of R- and S-isomers

Correlation between Aspartate Aminotransferase/Alanine Aminotransferase and Prognosis of Hemophagocytic Lymphohistiocytosis in Children

Background Aspartate aminotransferase (AST) /alanine aminotransferase (ALT) is a novel indicator to evaluate the prognosis of acute critical illness in recent years. At present, AST/ALT has only been reported to evaluate the prognosis of hemophagocytic lymphohistiocytosis (HLH) in adults, while HLH in children has not been studied. Objective To explore the relationship between AST/ALT and clinical characteristics and its prognostic significance in children with HLH, so as to provide a theoretical basis for early clinical recognition and diagnosis of HLH in children. Methods A total of 128 hospitalized children diagnosed with HLH in the Affiliated Hospital of Zunyi Medical University from January 2013 to May 2022 were selected as the research objects, and the baseline data of children were collected through the electronic medical record system. The children were divided into the T1 group (AST/ALT≤1.57, n=43), T2 group (1.57<AST/ALT<3.22, n=42), and T3 group (AST/ALT≥3.22, n=43) according to the AST/ALT quantiles, and followed up by outpatient review and telephone follow-up once every 6 months from the time of discharge to 2022-06-01, with the termination event of death or loss of follow-up. Spearman rank correlation analysis was used to explore the correlation between AST/ALT and laboratory test results. The receiver operating characteristic (ROC) curve of laboratory indicators for predicting death in children with HLHwas plotted, the area under ROC curve (AUC) and optimal cut-off value were calculated. Kaplan-Meier method was used to plot survival curves to analyze the effect of different AST/ALT groupings on overall survival, and Log-rank test was used for comparison. Cox proportional risk model was used to explore the influencing factors of death in children with HLH. Results There were statistically significant differences in gender, PICU admission, treatment methods, incidence of respiratory failure and shock among the 3 groups (P<0.05). Lactate dehydrogenase, creatine kinase isoenzyme, serum ferritin and activated partial thromboplastin time in the T3 group were higher than those in the T1 and T2 groups, while the levels of albumin and fibrinogen in the T3 group were lower than those in the T1 and T2 groups (P<0.05). Na+ level in the T2 and T3 groups was lower than that in the T1 group, while C-reactive protein level was higher than that in the T1 group (P<0.05). Correlation analysis showed that AST/ALT was positively correlated with absolute neutrophil count (rs=0.182, P=0.040), C-reactive protein (rs=0.419, P<0.001), total bilirubin (rs=0.182, P=0.040), creatine kinase isoenzyme (rs=0.310, P<0.001), lactate dehydrogenase (rs=0.474, P<0.001), activated partial thromboplastin time (rs=0.316, P<0.001), serum ferritin (rs=0.311, P<0.001), and negatively correlated with albumin (rs=-0.352, P<0.001), fibrinogen (rs=-0.179, P=0.043), Ca2+ (rs=-0.259, P=0.003), Na+ (rs=-0.244, P=0.006). ROC curve results showed that the AUCs of C-reactive protein, lactate dehydrogenase, activated partial thromboplastin time, serum ferritin and fibrinogen were 0.560〔95%CI (0.451, 0.669) 〕, 0.666〔95%CI (0.560, 0.772) 〕, 0.605〔95%CI (0.499, 0.710) 〕, 0.724〔95%CI (0.626, 0.822) 〕, 0.648〔95%CI (0.551, 0.745) 〕 and 0.715〔95%CI (0.624, 0.807) 〕, respectively, with the optimal cutoff values of 82.08 mg/L, 40.5 U/L, 927.5 U/L, 53.95 s, 1 897 μg/L, and 1.45 g/L, respectively. The mortality rate in the T1, T2 and T3 groups was 14.0% (6/43), 33.3% (14/42) and 44.2% (19/43), respectively, with statistically significant differences (χ2=9.518, P=0.009). Multivariate Cox proportional hazard regression analysis showed that shock〔HR=4.24, 95%CI (2.09, 8.61), P<0.001〕, activated partial thromboplastin time ≥53.95 s〔HR=2.44, 95%CI (1.24, 4.81), P=0.010〕and serum ferritin ≥1 897 μg/L〔HR=3.05, 95%CI (1.02, 9.09), P=0.046〕were the risk factors for death in children. Conclusion HLH patients in children with higher AST/ALT have higher incidence of poor prognosis, shorter overall survival, and worse prognosis