51 research outputs found

    Leave-one-out Distinguishability in Machine Learning

    Full text link
    We introduce a new analytical framework to quantify the changes in a machine learning algorithm's output distribution following the inclusion of a few data points in its training set, a notion we define as leave-one-out distinguishability (LOOD). This problem is key to measuring data **memorization** and **information leakage** in machine learning, and the **influence** of training data points on model predictions. We illustrate how our method broadens and refines existing empirical measures of memorization and privacy risks associated with training data. We use Gaussian processes to model the randomness of machine learning algorithms, and validate LOOD with extensive empirical analysis of information leakage using membership inference attacks. Our theoretical framework enables us to investigate the causes of information leakage and where the leakage is high. For example, we analyze the influence of activation functions, on data memorization. Additionally, our method allows us to optimize queries that disclose the most significant information about the training data in the leave-one-out setting. We illustrate how optimal queries can be used for accurate **reconstruction** of training data.Comment: Fixed typo

    Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learning

    Full text link
    Repeated parameter sharing in federated learning causes significant information leakage about private data, thus defeating its main purpose: data privacy. Mitigating the risk of this information leakage, using state of the art differentially private algorithms, also does not come for free. Randomized mechanisms can prevent convergence of models on learning even the useful representation functions, especially if there is more disagreement between local models on the classification functions (due to data heterogeneity). In this paper, we consider a representation federated learning objective that encourages various parties to collaboratively refine the consensus part of the model, with differential privacy guarantees, while separately allowing sufficient freedom for local personalization (without releasing it). We prove that in the linear representation setting, while the objective is non-convex, our proposed new algorithm \DPFEDREP\ converges to a ball centered around the \emph{global optimal} solution at a linear rate, and the radius of the ball is proportional to the reciprocal of the privacy budget. With this novel utility analysis, we improve the SOTA utility-privacy trade-off for this problem by a factor of d\sqrt{d}, where dd is the input dimension. We empirically evaluate our method with the image classification task on CIFAR10, CIFAR100, and EMNIST, and observe a significant performance improvement over the prior work under the same small privacy budget. The code can be found in this link: https://github.com/shenzebang/CENTAUR-Privacy-Federated-Representation-Learning.Comment: ICLR 2023 revise

    Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks

    Full text link
    We analytically investigate how over-parameterization of models in randomized machine learning algorithms impacts the information leakage about their training data. Specifically, we prove a privacy bound for the KL divergence between model distributions on worst-case neighboring datasets, and explore its dependence on the initialization, width, and depth of fully connected neural networks. We find that this KL privacy bound is largely determined by the expected squared gradient norm relative to model parameters during training. Notably, for the special setting of linearized network, our analysis indicates that the squared gradient norm (and therefore the escalation of privacy loss) is tied directly to the per-layer variance of the initialization distribution. By using this analysis, we demonstrate that privacy bound improves with increasing depth under certain initializations (LeCun and Xavier), while degrades with increasing depth under other initializations (He and NTK). Our work reveals a complex interplay between privacy and depth that depends on the chosen initialization distribution. We further prove excess empirical risk bounds under a fixed KL privacy budget, and show that the interplay between privacy utility trade-off and depth is similarly affected by the initialization

    Initialization matters : privacy-utility analysis of overparameterized neural networks

    Get PDF
    We analytically investigate how over-parameterization of models in randomized machine learning algorithms impacts the information leakage about their training data. Specifically, we prove a privacy bound for the KL divergence between model distributions on worst-case neighboring datasets, and explore its dependence on the initialization, width, and depth of fully connected neural networks. We find that this KL privacy bound is largely determined by the expected squared gradient norm relative to model parameters during training. Notably, for the special setting of linearized network, our analysis indicates that the squared gradient norm (and therefore the escalation of privacy loss) is tied directly to the per-layer variance of the initialization distribution. By using this analysis, we demonstrate that privacy bound improves with increasing depth under certain initializations (LeCun and Xavier), while degrades with increasing depth under other initializations (He and NTK). Our work reveals a complex interplay between privacy and depth that depends on the chosen initialization distribution. We further prove excess empirical risk bounds under a fixed KL privacy budget, and show that the interplay between privacy utility trade-off and depth is similarly affected by the initialization

    Swashplateless-elevon Actuation for a Dual-rotor Tail-sitter VTOL UAV

    Full text link
    In this paper, we propose a novel swashplateless-elevon actuation (SEA) for dual-rotor tail-sitter vertical takeoff and landing (VTOL) unmanned aerial vehicles (UAVs). In contrast to the conventional elevon actuation (CEA) which controls both pitch and yaw using elevons, the SEA adopts swashplateless mechanisms to generate an extra moment through motor speed modulation to control pitch and uses elevons solely for controlling yaw, without requiring additional actuators. This decoupled control strategy mitigates the saturation of elevons' deflection needed for large pitch and yaw control actions, thus improving the UAV's control performance on trajectory tracking and disturbance rejection performance in the presence of large external disturbances. Furthermore, the SEA overcomes the actuation degradation issues experienced by the CEA when the UAV is in close proximity to the ground, leading to a smoother and more stable take-off process. We validate and compare the performances of the SEA and the CEA in various real-world flight conditions, including take-off, trajectory tracking, and hover flight and position steps under external disturbance. Experimental results demonstrate that the SEA has better performances than the CEA. Moreover, we verify the SEA's feasibility in the attitude transition process and fixed-wing-mode flight of the VTOL UAV. The results indicate that the SEA can accurately control pitch in the presence of high-speed incoming airflow and maintain a stable attitude during fixed-wing mode flight. Video of all experiments can be found in youtube.com/watch?v=Sx9Rk4Zf7sQComment: 8 pages, 13 figure

    Case report: Oxaliplatin-induced idiopathic non-cirrhotic portal hypertension: a case report and literature review

    Get PDF
    Oxaliplatin has become a widely used agent in neoadjuvant chemotherapy for gastrointestinal tract tumors and is an integral part of the therapeutic approach for managing colorectal cancer recurrences and metastases, resulting in a more favorable prognosis for patients. Nevertheless, oxaliplatin can give rise to idiopathic non-cirrhotic portal hypertension (INCPH). The emergence of INCPH can disrupt tumor chemotherapy and incite persistent adverse reactions in later stages, significantly complicating clinical management. Consequently, we have presented a case report of INCPH induced by oxaliplatin chemotherapy with the aim of advancing the diagnosis and treatment of this condition, with a particular focus on the clinical manifestations. This study has ascertained that the condition is primarily attributed to complications related to portal hypertension, such as gastrointestinal bleeding, splenomegaly, and hypersplenism. The pathological features primarily involve hepatic sinus dilation and congestion, portal obstruction, absence, stenosis, shunting, localized venous and perisinusoidal fibrosis, as well as hepatocellular atrophy. Treatment primarily concentrates on strategies typically employed for cirrhosis. Endoscopic ligation, sclerotherapy, and non-selective beta-blockers (NSBBs) can be selected to prevent and treat variceal hemorrhage. Transjugular intrahepatic portosystemic shunt (TIPS) and liver transplantation can also be chosen for severe cases. Notably, despite the timely discontinuation of oxaliplatin, most patients continue to experience disease progression, ultimately resulting in a poor prognosis due to either tumor advancement or the ongoing progression of portal hypertension. This emphasizes the importance for physicians to be aware of and consider the risk of INCPH when prescribing oxaliplatin

    ROS-Dependent Activation of Autophagy through the PI3K/Akt/mTOR Pathway Is Induced by Hydroxysafflor Yellow A-Sonodynamic Therapy in THP-1 Macrophages

    Get PDF
    Monocyte-derived macrophages participate in infaust inflammatory responses by secreting various types of proinflammatory factors, resulting in further inflammatory reactions in atherosclerotic plaques. Autophagy plays an important role in inhibiting inflammation; thus, increasing autophagy may be a therapeutic strategy for atherosclerosis. In the present study, hydroxysafflor yellow A-mediated sonodynamic therapy was used to induce autophagy and inhibit inflammation in THP-1 macrophages. Following hydroxysafflor yellow A-mediated sonodynamic therapy, autophagy was induced as shown by the conversion of LC3-II/LC3-I, increased expression of beclin 1, degradation of p62, and the formation of autophagic vacuoles. In addition, inflammatory factors were inhibited. These effects were blocked by Atg5 siRNA, the autophagy inhibitor 3-methyladenine, and the reactive oxygen species scavenger N-acetyl cysteine. Moreover, AKT phosphorylation at Ser473 and mTOR phosphorylation at Ser2448 decreased significantly after HSYA-SDT. These effects were inhibited by the PI3K inhibitor LY294002, the AKT inhibitor triciribine, the mTOR inhibitor rapamycin, mTOR siRNA, and N-acetyl cysteine. Our results demonstrate that HSYA-SDT induces an autophagic response via the PI3K/Akt/mTOR signaling pathway and inhibits inflammation by reactive oxygen species in THP-1 macrophages

    Effect of sarcopenia on survival of patients with cirrhosis: A meta-analysis

    Get PDF
    The association between sarcopenia and prognosis in patients with cirrhosis remains to be determined. In this study, we aimed to quantify the association between sarcopenia and the risk of mortality in patients with cirrhosis, by sex, underlying liver disease etiology, and severity of hepatic dysfunction.PubMed, Web of Science, EMBASE, and major scientific conference sessions were searched without language restriction through 13 January 2021 with additional manual search of bibliographies of relevant articles. Cohort studies of ?100 patients with cirrhosis and ?12 months of follow-up that evaluated the association between sarcopenia, muscle mass and the risk of mortality were included.22 studies with 6965 patients with cirrhosis were included. The pooled prevalence of sarcopenia in patients with cirrhosis was 37.5% overall (95% CI 32.4%-42.8%), higher in male patients, patients with alcohol associated liver disease (ALD), patients with CTP grade C, and when sarcopenia was defined in patients by lumbar 3- skeletal muscle index (L3-SMI). Sarcopenia was associated with the increased risk of mortality in patients with cirrhosis (adjusted-hazard ratio [aHR] 2.30, 95% CI 2.01-2.63), with similar findings in sensitivity analysis of cirrhosis patients without HCC (aHR 2.35, 95% CI 1.95-2.83) and in subgroup analysis by sex, liver disease etiology, and severity of hepatic dysfunction. The association between quantitative muscle mass index and mortality further supports the poor prognosis for patients with sarcopenia (aHR 0.95, 95% CI 0.93-0.98). There was no significant heterogeneity in all analyses.Sarcopenia was highly and independently associated with higher risk of mortality in patients with cirrhosis.The prevalence of sarcopenia and its association with death in patients with cirrhosis remain unclear. This meta-analysis indicated that sarcopenia affected about one-third of patients with cirrhosis and up to 50% in patients with ALD or Child's class C cirrhosis. Sarcopenia was independently associated with about 2-fold higher risk of mortality in patients with cirrhosis. The mortality rate increased with greater severity or longer period of having sarcopenia. Increasing awareness about the importance of sarcopenia in patients with cirrhosis among stakeholders must be prioritized
    • ÔÇŽ
    corecore