Watermarking Federated Deep Neural Network Models

Training DNN models is expensive in terms of computational power, collection of a large amount of labeled data, and human expertise. Thus, DNN models constitute intellectual property (IP) and business value for their owners. Embedding digital watermarks into model training allows model owners to later demonstrate ownership, which can effectively protect the IP of their models. Recently, federated learning has been proposed as a new framework for machine learning development, which distributes the training of a global deep neural network (DNN) model over a large number of participants. Therefore, federated learning is advantageous than traditional DNN training in terms of data privacy, computational resources and a distributed optimization. However, there is no prior work investigating a solution for watermarking federated DNN models. The main challenge is that the distributed training causes the separation of training data (on participants' side) and watermark set (on aggregator's side), which does not satisfy the condition of traditional watermarking techniques that requires both training data and watermark set to be stored in the same place. In this thesis, we introduce two novel federated watermarking approaches which can embed watermark into federated DNN models by backdooring with low communication and computational overhead. In our approaches, the embedding of watermark is completed by the aggregator while the training is done by participants. We prove that our approaches embed a watermark with a high accuracy (100%) while keeping the functionality of the model. Moreover, the embedded watermarks in DNN models are resistant to post-processing techniques. We also propose a new watermark generation method and evaluate its efficacy in terms of unremovability, model utility and computational cost aspects

Get Out of the Valley: Power-Efficient Address Mapping for GPUs

GPU memory systems adopt a multi-dimensional hardware structure to provide the bandwidth necessary to support 100s to 1000s of concurrent threads. On the software side, GPU-compute workloads also use multi-dimensional structures to organize the threads. We observe that these structures can combine unfavorably and create significant resource imbalance in the memory subsystem causing low performance and poor power-efficiency. The key issue is that it is highly application-dependent which memory address bits exhibit high variability. To solve this problem, we first provide an entropy analysis approach tailored for the highly concurrent memory request behavior in GPU-compute workloads. Our window-based entropy metric captures the information content of each address bit of the memory requests that are likely to co-exist in the memory system at runtime. Using this metric, we find that GPU-compute workloads exhibit entropy valleys distributed throughout the lower order address bits. This indicates that efficient GPU-address mapping schemes need to harvest entropy from broad address-bit ranges and concentrate the entropy into the bits used for channel and bank selection in the memory subsystem. This insight leads us to propose the Page Address Entropy (PAE) mapping scheme which concentrates the entropy of the row, channel and bank bits of the input address into the bank and channel bits of the output address. PAE maps straightforwardly to hardware and can be implemented with a tree of XOR-gates. PAE improves performance by 1.31 x and power-efficiency by 1.25 x compared to state-of-the-art permutation-based address mapping

Level-S2^2fM: Structure from Motion on Neural Level Set of Implicit Surfaces

This paper presents a neural incremental Structure-from-Motion (SfM) approach, Level-S$^2$fM. In our formulation, we aim at simultaneously learning coordinate MLPs for the implicit surfaces and the radiance fields, and estimating the camera poses and scene geometry, which is mainly sourced from the established keypoint correspondences by SIFT. Our formulation would face some new challenges due to inevitable two-view and few-view configurations at the beginning of incremental SfM pipeline for the optimization of coordinate MLPs, but we found that the strong inductive biases conveying in the 2D correspondences are feasible and promising to avoid those challenges by exploiting the relationship between the ray sampling schemes used in volumetric rendering and the sphere tracing of finding the zero-level set of implicit surfaces. Based on this, we revisit the pipeline of incremental SfM and renew the key components of two-view geometry initialization, the camera pose registration, and the 3D points triangulation, as well as the Bundle Adjustment in a novel perspective of neural implicit surfaces. Because the coordinate MLPs unified the scene geometry in small MLP networks, our Level-S$^2$fM treats the zero-level set of the implicit surface as an informative top-down regularization to manage the reconstructed 3D points, reject the outlier of correspondences by querying SDF, adjust the estimated geometries by NBA (Neural BA), finally yielding promising results of 3D reconstruction. Furthermore, our Level-S$^2$fM alleviated the requirement of camera poses for neural 3D reconstruction.Comment: under revie

Flexible Differentially Private Vertical Federated Learning with Adaptive Feature Embeddings

The emergence of vertical federated learning (VFL) has stimulated concerns about the imperfection in privacy protection, as shared feature embeddings may reveal sensitive information under privacy attacks. This paper studies the delicate equilibrium between data privacy and task utility goals of VFL under differential privacy (DP). To address the generality issue of prior arts, this paper advocates a flexible and generic approach that decouples the two goals and addresses them successively. Specifically, we initially derive a rigorous privacy guarantee by applying norm clipping on shared feature embeddings, which is applicable across various datasets and models. Subsequently, we demonstrate that task utility can be optimized via adaptive adjustments on the scale and distribution of feature embeddings in an accuracy-appreciative way, without compromising established DP mechanisms. We concretize our observation into the proposed VFL-AFE framework, which exhibits effectiveness against privacy attacks and the capacity to retain favorable task utility, as substantiated by extensive experiments

Specification Overfitting in Artificial Intelligence

Machine learning (ML) and artificial intelligence (AI) approaches are often criticized for their inherent bias and for their lack of control, accountability, and transparency. Consequently, regulatory bodies struggle with containing this technology's potential negative side effects. High-level requirements such as fairness and robustness need to be formalized into concrete specification metrics, imperfect proxies that capture isolated aspects of the underlying requirements. Given possible trade-offs between different metrics and their vulnerability to over-optimization, integrating specification metrics in system development processes is not trivial. This paper defines specification overfitting, a scenario where systems focus excessively on specified metrics to the detriment of high-level requirements and task performance. We present an extensive literature survey to categorize how researchers propose, measure, and optimize specification metrics in several AI fields (e.g., natural language processing, computer vision, reinforcement learning). Using a keyword-based search on papers from major AI conferences and journals between 2018 and mid-2023, we identify and analyze 74 papers that propose or optimize specification metrics. We find that although most papers implicitly address specification overfitting (e.g., by reporting more than one specification metric), they rarely discuss which role specification metrics should play in system development or explicitly define the scope and assumptions behind metric formulations.Comment: 40 pages, 2 figure

Volumetric Wireframe Parsing from Neural Attraction Fields

The primal sketch is a fundamental representation in Marr's vision theory, which allows for parsimonious image-level processing from 2D to 2.5D perception. This paper takes a further step by computing 3D primal sketch of wireframes from a set of images with known camera poses, in which we take the 2D wireframes in multi-view images as the basis to compute 3D wireframes in a volumetric rendering formulation. In our method, we first propose a NEural Attraction (NEAT) Fields that parameterizes the 3D line segments with coordinate Multi-Layer Perceptrons (MLPs), enabling us to learn the 3D line segments from 2D observation without incurring any explicit feature correspondences across views. We then present a novel Global Junction Perceiving (GJP) module to perceive meaningful 3D junctions from the NEAT Fields of 3D line segments by optimizing a randomly initialized high-dimensional latent array and a lightweight decoding MLP. Benefitting from our explicit modeling of 3D junctions, we finally compute the primal sketch of 3D wireframes by attracting the queried 3D line segments to the 3D junctions, significantly simplifying the computation paradigm of 3D wireframe parsing. In experiments, we evaluate our approach on the DTU and BlendedMVS datasets with promising performance obtained. As far as we know, our method is the first approach to achieve high-fidelity 3D wireframe parsing without requiring explicit matching.Comment: Technical report; Video can be found at https://youtu.be/qtBQYbOpVp

Effect of angiotensin receptor neprilysin inhibitors on left atrial remodeling and prognosis in heart failure

Aims The angiotensin receptor–neprilysin inhibitor (ARNI), sacubitril/valsartan, confers additional protective effects compared with angiotensin-converting enzyme inhibitors/angiotensin II receptor blockers (ACEIs/ARBs) in terms of reversed left ventricular (LV) remodelling and improves the prognosis of patients with heart failure (HF). However, few studies have examined the effects of ARNI on the left atrium. Accordingly, this study compared the effects of ARNI and ACEI/ARB on left atrial (LA) remodelling in heart failure with reduced ejection fraction (HFrEF). Methods and results This was a single-centre retrospective study of patients with HFrEF hospitalized at the First Affiliated Hospital of Dalian Medical University between 26 February 2016 and 8 July 2020. Patients were classified into ARNI and ACEI/ARB groups and further subgroups based on the left atrial volume index (LAVI): mildly abnormal (29 mL/m2 ≤ LAVI < 34 mL/m2), moderately abnormal (34 mL/m2 ≤ LAVI < 40 mL/m2), and severely abnormal (LAVI ≥ 40 mL/m2). The primary endpoint was changes in LA parameters by echocardiography. The secondary endpoint was all-cause mortality. A total of 336 patients (mean age: 64.11 ± 12.86, 30.06% female) were included. Except those lost to follow-up, 274 HFrEF patients remained, with 144 cases in the ARNI group and 130 cases in the ACEI/ARB group. Greater reductions from baseline were seen with ARNI in LA diameter (LAD) (P = 0.013, t-test), superior and LA superior–inferior diameter (LASID) (P < 0.0001), LA transverse diameter (LATD) (P < 0.0001), LA volume (LAV) (P < 0.0001), LAVI (P < 0.0001), and LA sphericity index (LASI) (P < 0.0001). Over a mean follow-up of 19.40 months, 97 patients (67.3%) in the ARNI group and 29 patients (22.3%) in the ACEI/ARB group showed LA reverse remodelling (LARR). Kaplan–Meier analysis showed significantly lower overall mortality in the ARNI group compared with the ACEI/ARB group (P = 0.048, log-rank test). The mildly abnormal LAVI group of ARNI patients showed a reduction in mortality compared with ACEI/ARB patients (P = 0.044). However, no significant difference was observed for the moderately abnormal (P = 0.571) or severely abnormal LAVI groups (P = 0.609), suggesting that early initiation of ARNI was associated with a better prognosis. Conclusions In this proof-of-concept study, ARNI use showed greater effects on LARR and was associated with a better prognosis compared with ACEI/ARB use in HFrEF. Early initiation of ARNI in the HF disease process may produce greater benefit, but this needs to be confirmed in future studies

Reverse atrial remodeling in heart failure with recovered ejection fraction

Background Heart failure with recovered ejection fraction (HFrecEF) has been a newly recognized entity since 2020. However, the concept has primarily focused on left ventricular ejection fraction improvement, with less focus on the recovery of the left atrium. In this study, we investigated changes in left atrial (LA) echocardiographic indices in HFrecEF. Methods and Results An inpatient cohort with heart failure with reduced ejection fraction (HFrEF) was identified retrospectively and followed up prospectively in a single tertiary hospital. The enrolled patients were classified into HFrecEF and persistent HFrEF groups. Alternations in LA parameters by echocardiography were calculated. The primary outcome was a composite of cardiovascular death or heart failure rehospitalization. A total of 699 patients were included (HFrecEF: n=228; persistent HFrEF: n=471). Compared with persistent HFrEF, the HFrecEF group had greater reductions in LA diameter, LA transverse diameter, LA superior–inferior diameter, LA volume, and LA volume index but not in LA sphericity index. Cox regression analysis showed that the HFrecEF group experienced lower risks of prespecified end points than the persistent HFrEF group after adjusting for confounders. Additionally, 136 (59.6%) and 62 (13.0%) patients showed LA reverse remodeling (LARR) for the HFrecEF and persistent HFrEF groups, respectively. Among the HFrecEF subgroup, patients with LARR had better prognosis compared with those without LARR. Multivariate logistic analysis demonstrated that age and coronary heart disease were 2 independent negative predictors for LARR. Conclusions In HFrecEF, both left ventricular systolic function and LA structure remodeling were improved. Patients with HFrecEF with LARR had improved clinical outcomes, indicating that the evaluation of LA size provides a useful biomarker for risk stratification of heart failure