Generalizing DP-SGD with shuffling and batch clipping

Classical differential private DP-SGD implements individual clipping with random subsampling, which forces a mini-batch SGD approach. We provide a general differential private algorithmic framework that goes beyond DP-SGD and allows any possible first order optimizers (e.g., classical SGD and momentum based SGD approaches) in combination with batch clipping, which clips an aggregate of computed gradients rather than summing clipped gradients (as is done in individual clipping). The framework also admits sampling techniques beyond random subsampling such as shuffling. Our DP analysis follows the f -DP approach and introduces a new proof technique based on a slightly stronger adversarial model which allows us to derive simple closed form expressions and to also analyse group privacy. In particular, for E epochs work and groups of size g, we show a√gE DP dependency for batch clipping with shuffling

Besting the black-box: Barrier zones for adversarial example defense

Adversarial machine learning defenses have primarily been focused on mitigating static, white-box attacks. However, it remains an open question whether such defenses are robust under an adaptive black-box adversary. In this paper, we specifically focus on the black-box threat model and make the following contributions: First we develop an enhanced adaptive black-box attack which is experimentally shown to be ≥ 30% more effective than the original adaptive black-box attack proposed by Papernot et al. For our second contribution, we test 10 recent defenses using our new attack and propose our own black-box defense (barrier zones). We show that our defense based on barrier zones offers significant improvements in security over state-of-the-art defenses. This improvement includes greater than 85% robust accuracy against black-box boundary attacks, transfer attacks and our new adaptive black-box attack, for the datasets we study. For completeness, we verify our claims through extensive experimentation with 10 other defenses using three adversarial models (14 different black-box attacks) on two datasets (CIFAR-10 and Fashion-MNIST)

On the Tightness of the Moment Accountant for DP-SGD

In order to provide differential privacy, Gaussian noise with standard deviation σ is added to local SGD updates after performing a clipping operation in Differential Private SGD (DP-SGD). By non-trivially improving the moment account method we prove a closed form (ϵ, δ)-DP guarantee: DP-SGD is (ϵ ≤ 1/2, δ = 1/N )-DP if σ = p2(ϵ + ln(1/δ))/ϵ with T at least ≈ 2k2/ϵ and (2/e)2k2 − 1/2 ≥ ln(N ), where T is the total number of rounds, and K = kN is the total number of gradient computations where k measures K in number of epochs of size N of the local data set. We prove that our expression is close to tight in that if T is more than a constant factor ≈ 8 smaller than the lower bound ≈ 2k2/ϵ, then the (ϵ, δ)-DP guarantee is violated. Choosing the smallest possible value T ≈ 2k2/ϵ not only leads to a close to tight DP guarantee, but also minimizes the total number of communicated updates and this means that the least amount of noise is aggregated into the global model and in addition accuracy is optimized as confirmed by simulations

A unified convergence analysis for shuffling-type gradient methods

In this paper, we propose a unified convergence analysis for a class of generic shuffling-type gradient methods for solving finite-sum optimization problems. Our analysis works with any sampling without replacement strategy and covers many known variants such as randomized reshuffling, deterministic or randomized single permutation, and cyclic and incremental gradient schemes. We focus on two different settings: strongly convex and nonconvex problems, but also discuss the non-strongly convex case. Our main contribution consists of new non-asymptotic and asymptotic convergence rates for a wide class of shuffling-type gradient methods in both nonconvex and convex settings. We also study uniformly randomized shuffling variants with different learning rates and model assumptions. While our rate in the nonconvex case is new and significantly improved over existing works under standard assumptions, the rate on the strongly convex one matches the existing best-known rates prior to this paper up to a constant factor without imposing a bounded gradient condition. Finally, we empirically illustrate our theoretical results via two numerical examples: nonconvex logistic regression and neural network training examples. As byproducts, our results suggest some appropriate choices for diminishing learning rates in certain shuffling variants

Secure Remote Attestation with Strong Key Insulation Guarantees

Recent years have witnessed a trend of secure processor design in both academia and industry. Secure processors with hardware-enforced isolation can be a solid foundation of cloud computation in the future. However, due to recent side-channel attacks, the commercial secure processors failed to deliver the promises of a secure isolated execution environment. Sensitive information inside the secure execution environment always gets leaked via side channels. This work considers the most powerful software-based side-channel attackers, i.e., an All Digital State Observing (ADSO) adversary who can observe all digital states, including all digital states in secure enclaves. Traditional signature schemes are not secure in ADSO adversarial model. We introduce a new cryptographic primitive called One-Time Signature with Secret Key Exposure (OTS-SKE), which ensures no one can forge a valid signature of a new message or nonce even if all secret session keys are leaked. OTS-SKE enables us to sign attestation reports securely under the ADSO adversary. We also minimize the trusted computing base by introducing a secure co-processor into the system, and the interaction between the secure co-processor and the attestation processor is unidirectional. That is, the co-processor takes no inputs from the processor and only generates secret keys for the processor to fetch. Our experimental results show that the signing of OTS-SKE is faster than that of Elliptic Curve Digital Signature Algorithm (ECDSA) used in Intel SGX

Bilinear map based one-time signature scheme with secret key exposure

Dijk et al. [6] presents Remote Attestation (RA) for secure processor technology which is secure in the presence of an All Digital State Observing (ADSO) adversary. The scheme uses a combination of hardware security primitives and design principles together with a new cryptographic primitive called a Public Key Session based One-Time Signature Scheme with Secret Key Exposure (OTS-SKE). [6] shows a hash based realization of OTS-SKE which is post quantum secure but suffers long 8.704 KB signatures for 128-bit quantum security or 256-bit classical security. From a classical cryptographic perspective we complete the picture by introducing a bilinear map based OTS-SKE with short 0.125 KB signatures, 65 times shorter, and for which the security reduces to the Computational Diffie-Hellman Problem (CDHP) – at the cost of a 9× longer initialization phase in the RA scheme if implemented in software (this can be improved with appropriate elliptic curve hardware acceleration). Signing takes 560 ms at most 60% of the > 936 ms needed for the hash based scheme

Autonomous secure remote attestation even when all used and to be used digital keys leak

We provide a new remote attestation scheme for secure processor technology, which is secure in the presence of an All Digital State Observing (ADSO) adversary. To accomplish this, we obfuscate session signing keys using a silicon Physical Unclonable Function (PUF) with an extended interface that combines the LPN-PUF concept with a repetition code for small failure probabilities, and we introduce a new signature scheme that only needs a message dependent subset of a session signing key for computing a signature and whose signatures cannot be successfully forged even if one subset per session signing key leaks. Our solution for remote attestation shows that results computed by enclaves can be properly verified even when an ADSO-adversary is present. For $N=2^l$ sessions, implementation results show that signing takes $934.9+0.6\cdot l$ ms and produces a signature of $8.2+0.03\cdot l$ KB, and verification by a remote user takes $118.2+0.4\cdot l$ ms. During initialization, generation of all session keys takes $819.3 \cdot N$ ms and corresponding storage is $3 \cdot 10^{-5} + 0.12 \cdot N$ MB

Faktor Penguat Pada Peningkatan Kinerja Karyawan PT. Gading Murni Surabaya

The purpose of this study is to determine which factors as an amplifier to improve the performance of employees of PT. Gading Murni Surabaya, between the leadership style and compensation received by employees. This study uses a survey approach by collecting data using a questionnaire of 50 respondents then analyzed using quantitative methods. The regession equation is Y = 8.009 + 0,223 X1 + 0,240 X2. The results of this study concluded that the leadership style and compensation variables had a corrected item total correlation value exceeding r table = 0.284 and the reliability test of the leadership style variable, and the alpha cronbach's compensation results exceeded 0.060, which means that the variable was valid and reliable. Leadership and compensation styles also simultaneously have a significant effect on employee performance. And the independent variable that has the largest beta coefficient is the compensation variable (X2) with a beta coefficient of 0.240.   Keywords : Leadership style, Compesation and Employee Performance