Homomorphically encrypted gradient descent algorithms for quadratic programming

In this paper, we evaluate the different fully homomorphic encryption schemes, propose an implementation, and numerically analyze the applicability of gradient descent algorithms to solve quadratic programming in a homomorphic encryption setup. The limit on the multiplication depth of homomorphic encryption circuits is a major challenge for iterative procedures such as gradient descent algorithms. Our analysis not only quantifies these limitations on prototype examples, thus serving as a benchmark for future investigations, but also highlights additional trade-offs like the ones pertaining the choice of gradient descent or accelerated gradient descent methods, opening the road for the use of homomorphic encryption techniques in iterative procedures widely used in optimization based control. In addition, we argue that, among the available homomorphic encryption schemes, the one adopted in this work, namely CKKS, is the only suitable scheme for implementing gradient descent algorithms. The choice of the appropriate step size is crucial to the convergence of the procedure. The paper shows firsthand the feasibility of homomorphically encrypted gradient descent algorithms

Privacy-Preserving Gaussian Process Regression -- A Modular Approach to the Application of Homomorphic Encryption

Much of machine learning relies on the use of large amounts of data to train models to make predictions. When this data comes from multiple sources, for example when evaluation of data against a machine learning model is offered as a service, there can be privacy issues and legal concerns over the sharing of data. Fully homomorphic encryption (FHE) allows data to be computed on whilst encrypted, which can provide a solution to the problem of data privacy. However, FHE is both slow and restrictive, so existing algorithms must be manipulated to make them work efficiently under the FHE paradigm. Some commonly used machine learning algorithms, such as Gaussian process regression, are poorly suited to FHE and cannot be manipulated to work both efficiently and accurately. In this paper, we show that a modular approach, which applies FHE to only the sensitive steps of a workflow that need protection, allows one party to make predictions on their data using a Gaussian process regression model built from another party's data, without either party gaining access to the other's data, in a way which is both accurate and efficient. This construction is, to our knowledge, the first example of an effectively encrypted Gaussian process

Privacy-Preserving intrusion detection over network data

Effective protection against cyber-attacks requires constant monitoring and analysis of system data such as log files and network packets in an IT infrastructure, which may contain sensitive information. To this end, security operation centers (SOC) are established to detect, analyze, and respond to cyber-security incidents. Security officers at SOC are not necessarily trusted with handling the content of the sensitive and private information, especially in case when SOC services are outsourced as maintaining in-house expertise and capability in cyber-security is expensive. Therefore, an end-to-end security solution is needed for the system data. SOC often utilizes detection models either for known types of attacks or for an anomaly and applies them to the collected data to detect cyber-security incidents. The models are usually constructed from historical data that contains records pertaining to attacks and normal functioning of the IT infrastructure under monitoring; e.g., using machine learning techniques. SOC is also motivated to keep its models confidential for three reasons: i) to capitalize on the models that are its propriety expertise, ii) to protect its detection strategies against adversarial machine learning, in which intelligent and adaptive adversaries carefully manipulate their attack strategy to avoid detection, and iii) the model might have been trained on sensitive information, whereby revealing the model can violate certain laws and regulations. Therefore, detection models are also private. In this dissertation, we propose a scenario in which privacy of both system data and detection models is protected and information leakage is either prevented altogether or quantifiably decreased. Our main approach is to provide an end-to-end encryption for system data and detection models utilizing lattice-based cryptography that allows homomorphic operations over the encrypted data. Assuming that the detection models are previously obtained from training data by SOC, we apply the models to system data homomorphically, whereby the model is encrypted. We take advantage of three different machine learning algorithms to extract intrusion models by training historical data. Using different data sets (two recent data sets, and one outdated but widely used in the intrusion detection literature), the performance of each algorithm is evaluated via the following metrics: i) the time that takes to extract the rules, ii) the time that takes to apply the rules on data homomorphically, iii) the accuracy of the rules in detecting intrusions, and iv) the number of rules. Our experiments demonstrates that the proposed privacy-preserving intrusion detection system (IDS) is feasible in terms of execution times and reliable in terms of accurac

Privacy Preserving Inference for Deep Neural Networks:Optimizing Homomorphic Encryption for Efficient and Secure Classification

The application of machine learning in healthcare, financial, social media, and other sensitive sectors not only involves high accuracy but privacy as well. Due to the emergence of the Cloud as a computation and one-to-many access paradigm; training and classification/inference tasks have been outsourced to Cloud. However, its usage is limited due to legal and ethical constraints regarding privacy. In this work, we propose a privacy-preserving neural networks-based classification model based on Homomorphic Encryption (HE) where the user can send an encrypted instance to the cloud and receive an encrypted inference from it to preserve the user’s query privacy. In contrast to existing works, we demonstrate the realistic limitations of HE for privacy-preserving machine learning by changing its parameters for enhanced security and accuracy. We showcase scenarios where the choice of HE parameters impedes accurate classification and present an optimized setting for achieving reliable classification. We present several results to demonstrate its effectiveness using MNIST dataset with highly improved inference time for a query as compared to the state of the art

Secure MultiParty Protocol for Differentially-Private Data Release

In the era where big data is the new norm, a higher emphasis has been placed on models which guarantees the release and exchange of data. The need for privacy-preserving data arose as more sophisticated data-mining techniques led to breaches of sensitive information. In this thesis, we present a secure multiparty protocol for the purpose of integrating multiple datasets simultaneously such that the contents of each dataset is not revealed to any of the data owners, and the contents of the integrated data do not compromise individual’s privacy. We utilize privacy by simulation to prove that the protocol is privacy-preserving, and we show that the output data satisfies ϵ-differential privacy

정보 보호 기계 학습의 암호학 기반 기술: 근사 동형 암호와 부호 기반 암호

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2021. 2. 노종선.In this dissertation, three main contributions are given as; i) a protocol of privacy-preserving machine learning using network resources, ii) the development of approximate homomorphic encryption that achieves less error and high-precision bootstrapping algorithm without compromising performance and security, iii) the cryptanalysis and the modification of code-based cryptosystems: cryptanalysis on IKKR cryptosystem and modification of the pqsigRM, a digital signature scheme proposed to the post-quantum cryptography (PQC) standardization of National Institute of Standards and Technology (NIST). The recent development of machine learning, cloud computing, and blockchain raises a new privacy problem; how can one outsource computation on confidential data? Moreover, as research on quantum computers shows success, the need for PQC is also emerging. Multi-party computation (MPC) is the cryptographic protocol that makes computation on data without revealing it. Since MPC is designed based on homomorphic encryption (HE) and PQC, research on designing efficient and safe HE and PQC is actively being conducted. First, I propose a protocol for privacy-preserving machine learning (PPML) that replaces bootstrapping of homomorphic encryption with network resources. In general, the HE ciphertext has a limited depth of circuit that can be calculated, called the level of a ciphertext. We call bootstrapping restoring the level of ciphertext that has exhausted its level through a method such as homomorphic decryption. Bootstrapping of homomorphic encryption is, in general, very expensive in time and space. However, when deep computations like deep learning are performed, it is required to do bootstrapping. In this protocol, both the client's message and servers' intermediate values are kept secure, while the client's computation and communication complexity are light. Second, I propose an improved bootstrapping algorithm for the CKKS scheme and a method to reduce the error by homomorphic operations in the CKKS scheme. The Cheon-Kim-Kim-Song (CKKS) scheme (Asiacrypt '17) is one of the highlighted fully homomorphic encryption (FHE) schemes as it is efficient to deal with encrypted real numbers, which are the usual data type for many applications such as machine learning. However, the precision drop due to the error growth is a drawback of the CKKS scheme for data processing. I propose a method to achieve high-precision approximate FHE using the following two methods .First, I apply the signal-to-noise ratio (SNR) concept and propose methods to maximize SNR by reordering homomorphic operations in the CKKS scheme. For that, the error variance is minimized instead of the upper bound of error when we deal with the encrypted data. Second, from the same perspective of minimizing error variance, I propose a new method to find the approximate polynomials for the CKKS scheme. The approximation method is especially applied to the CKKS scheme's bootstrapping, where we achieve bootstrapping with smaller error variance compared to the prior arts. In addition to the above variance-minimizing method, I cast the problem of finding an approximate polynomial for a modulus reduction into an L2-norm minimization problem. As a result, I find an approximate polynomial for the modulus reduction without using the sine function, which is the upper bound for the polynomial approximation of the modulus reduction. By using the proposed method, the constraint of q = O(m^{3/2}) is relaxed as O(m), and thus the level loss in bootstrapping can be reduced. The performance improvement by the proposed methods is verified by implementation over HE libraries, that is, HEAAN and SEAL. The implementation shows that by reordering homomorphic operations and using the proposed polynomial approximation, the reliability of the CKKS scheme is improved. Therefore, the quality of services of various applications using the proposed CKKS scheme, such as PPML, can be improved without compromising performance and security. Finally, I propose an improved code-based signature scheme and cryptanalysis of code-based cryptosystems. A novel code-based signature scheme with small parameters and an attack algorithm on recent code-based cryptosystems are presented in this dissertation. This scheme is based on a modified Reed-Muller (RM) code, which reduces the signing complexity and key size compared with existing code-based signature schemes. The proposed scheme has the advantage of the pqsigRM decoder and uses public codes that are more difficult to distinguish from random codes. I use (U, U+V) -codes with the high-dimensional hull to overcome the disadvantages of code-based schemes. The proposed a decoder which efficiently samples from coset elements with small Hamming weight for any given syndrome. The proposed signature scheme resists various known attacks on RM code-based cryptography. For 128 bits of classical security, the signature size is 4096 bits, and the public key size is less than 1 MB. Recently, Ivanov, Kabatiansky, Krouk, and Rumenko (IKKR) proposed three new variants of the McEliece cryptosystem (CBCrypto 2020, affiliated with Eurocrypt 2020). This dissertation shows that one of the IKKR cryptosystems is equal to the McEliece cryptosystem. Furthermore, a polynomial-time attack algorithm for the other two IKKR cryptosystems is proposed. The proposed attack algorithm utilizes the linearity of IKKR cryptosystems. Also, an implementation of the IKKR cryptosystems and the proposed attack is given. The proposed attack algorithm finds the plaintext within 0.2 sec, which is faster than the elapsed time for legitimate decryption.본 논문은 크게 다음의 세 가지의 기여를 포함한다. i) 네트워크를 활용해서 정보 보호 딥러닝을 개선하는 프로토콜 ii) 근사 동형 암호에서 보안성과 성능의 손해 없이 에러를 낮추고 높은 정확도로 부트스트래핑 하는 방법 iii) IKKR 암호 시스템과 pqsigRM 등 부호 기반 암호를 공격하는 방법과 효율적인 부호 기반 전자 서명 시스템. 근래의 기계학습과 블록체인 기술의 발전으로 인해서 기밀 데이터에 대한 연산을 어떻게 외주할 수 있느냐에 대한 새로운 보안 문제가 대두되고 있다. 또한, 양자 컴퓨터에 관한 연구가 성공을 거듭하면서, 이를 이용한 공격에 저항하는 포스트 양자 암호의 필요성 또한 커지고 있다. 다자간 컴퓨팅은 데이터를 공개하지 않고 데이터에 대한 연산을 수행할 수 있도록 하는 암호학적 프로토콜의 총칭이다. 다자간 컴퓨팅은 동형 암호와 포스트 양자 암호에 기반하고 있으므로, 효율적인 동형 암호와 포스트 양자 암호에 관한 연구가 활발하게 수행되고 있다. 동형 암호는 암호화된 데이터에 대한 연산이 가능한 특수한 암호화 알고리즘이다. 일반적으로 동형 암호의 암호문에 대해서 수행 가능한 연산의 깊이가 정해져 있으며, 이를 암호문의 레벨이라고 칭한다. 레벨을 모두 소비한 암호문의 레벨을 다시 복원하는 과정을 부트스트래핑 (bootstrapping)이라고 칭한다. 일반적으로 부트스트래핑은 매우 오래 걸리는 연산이며 시간 및 공간 복잡도가 크다. 그러나, 딥러닝과 같이 깊이가 큰 연산을 수행하는 경우 부트스트래핑이 필수적이다. 본 논문에서는 정보 보호 기계학습을 위한 새로운 프로토콜을 제안한다. 이 프로토콜에서는 입력 메시지와 더불어 신경망의 중간값들 또한 안전하게 보호된다. 그러나 여전히 사용자의 통신 및 연산 복잡도는 낮게 유지된다. Cheon, Kim, Kim 그리고 Song (CKKS)가 제안한 암호 시스템 (Asiacrypt 17)은 기계학습 등에서 가장 널리 쓰이는 데이터인 실수를 효율적으로 다룰 수 있으므로 가장 촉망받는 완전 동형 암호 시스템이다. 그러나, 오류의 증폭과 전파가 CKKS 암호 시스템의 가장 큰 단점이다. 이 논문에서는 아래의 기술을 활용하여 CKKS 암호 시스템의 오류를 줄이는 방법을 제안하며, 이는 근사 동형 암호에 일반화하여 적용할 수 있다. 첫째, 신호 대비 잡음 비 (signal-to-noise ratio, SNR)의 개념을 도입하여, SNR를 최대화하도록 연산의 순서를 재조정한다. 그러기 위해서는, 오류의 최대치 대신 분산이 최소화되어야 하며, 이를 관리해야 한다. 둘째, 오류의 분산을 최소화한다는 같은 관점에서 새로운 다항식 근사 방법을 제안한다. 이 근사 방법은 특히, CKKS 암호 시스템의 부트스트래핑에 적용되었으며, 종래 기술보다 더 낮은 오류를 달성한다. 위의 방법에 더하여, 근사 다항식을 구하는 문제를 L2-norm 최소화 문제로 치환하는 방법을 제안한다. 이를 통해서 사인 함수의 도입 없이 근사 다항식을 구하는 방법을 제안한다. 제안된 방법을 사용하면, q=O(m^{3/2})라는 제약을 q=O(m)으로 줄일 수 있으며, 부트스트래핑에 필요한 레벨 소모를 줄일 수 있다. 성능 향상은 HEAAN과 SEAL 등의 동형 암호 라이브러리를 활용한 구현을 통해 증명했으며, 구현을 통해서 연산 재정렬과 새로운 부트스트래핑이 CKKS 암호 시스템의 성능을 향상함을 확인했다. 따라서, 보안성과 성능의 타협 없이 근사 동형 암호를 사용하는 서비스의 질을 향상할 수 있다. 양자 컴퓨터를 활용하여 전통적인 공개키 암호를 공격하는 효율적인 알고리즘이 공개되면서, 포스트 양자 암호에 대한 필요성이 증대했다. 부호 기반 암호는 포스트 양자 암호로써 널리 연구되었다. 작은 키 크기를 갖는 새로운 부호 기반 전자 서명 시스템과 부호 기반 암호를 공격하는 방법이 논문에 제안되어 있다. pqsigRM이라 명명한 전자 서명 시스템이 그것이다. 이 전자 서명 시스템은 수정된 Reed-Muller (RM) 부호를 활용하며, 서명의 복잡도와 키 크기를 종래 기술보다 많이 줄인다. pqsigRM은 hull의 차원이 큰 (U, U+V) 부호와 이의 복호화를 이용하여, 서명에서 큰 이득이 있다. 이 복호화 알고리즘은 주어진 모든 코셋 (coset)의 원소에 대하여 작은 헤밍 무게를 갖는 원소를 반환한다. 또한, 수정된 RM 부호를 이용하여, 알려진 모든 공격에 저항한다. 128비트 안정성에 대해서 서명의 크기는 4096 비트이고, 공개 키의 크기는 1MB보다 작다. 최근, Ivanov, Kabatiansky, Krouk, 그리고 Rumenko (IKKR)가 McEliece 암호 시스템의 세 가지 변형을 발표했다 (CBCrypto 2020, Eurocrypt 2020와 함께 진행). 본 논문에서는 IKKR 암호 시스템중 하나가 McEliece 암호 시스템과 동치임을 증명한다. 또한 나머지 IKKR 암호 시스템에 대한 다항 시간 공격을 제안한다. 제안하는 공격은 IKKR 암호 시스템의 선형성을 활용한다. 또한, 이 논문은 제안한 공격의 구현을 포함하며, 제안된 공격은 0.2초 이내에 메시지를 복원하고, 이는 정상적인 복호화보다 빠른 속도이다.Contents Abstract i Contents iv List of Tables ix List of Figures xi 1 Introduction 1 1.1 Homomorphic Encryption and Privacy-Preserving Machine Learning 4 1.2 High-Precision CKKS Scheme and Its Bootstrapping 5 1.2.1 Near-Optimal Bootstrapping of the CKKS Scheme Using Least Squares Method 6 1.2.2 Variance-Minimizing and Optimal Bootstrapping of the CKKS Scheme 8 1.3 Efficient Code-Based Signature Scheme and Cryptanalysis of the Ivanov-Kabatiansky-Krouk-Rumenko Cryptosystems 10 1.3.1 Modified pqsigRM: An Efficient Code-Based Signature Scheme 11 1.3.2 Ivanov-Kabatiansky-Krouk-Rumenko Cryptosystems and Its Equality 13 1.4 Organization of the Dissertation 14 2 Preliminaries 15 2.1 Basic Notation 15 2.2 Privacy-Preserving Machine Learning and Security Terms 16 2.2.1 Privacy-Preserving Machine Learning and Security Terms 16 2.2.2 Privacy-Preserving Machine Learning 17 2.3 The CKKS Scheme and Its Bootstrapping 18 2.3.1 The CKKS Scheme 18 2.3.2 CKKS Scheme in RNS 22 2.3.3 Bootstrapping of the CKKS Scheme 24 2.3.4 Statistical Characteristics of Modulus Reduction and Failure Probability of Bootstrapping of the CKKS Scheme 26 2.4 Approximate Polynomial and Signal-to-Noise Perspective for Approximate Homomorphic Encryption 27 2.4.1 Chebyshev Polynomials 27 2.4.2 Signal-to-Noise Perspective of the CKKS Scheme 28 2.5 Preliminary for Code-Based Cryptography 29 2.5.1 The McEliece Cryptosystem 29 2.5.2 CFS Signature Scheme 30 2.5.3 ReedMuller Codes and Recursive Decoding 31 2.5.4 IKKR Cryptosystems 33 3 Privacy-Preserving Machine Learning via FHEWithout Bootstrapping 37 3.1 Introduction 37 3.2 Information Theoretic Secrecy and HE for Privacy-Preserving Machine Learning 38 3.2.1 The Failure Probability of Ordinary CKKS Bootstrapping 39 3.3 Comparison With Existing Methods 43 3.3.1 Comparison With the Hybrid Method 43 3.3.2 Comparison With FHE Method 44 3.4 Comparison for Evaluating Neural Network 45 4 High-Precision Approximate Homomorphic Encryption and Its Bootstrapping by Error Variance Minimization and Convex Optimization 50 4.1 Introduction 50 4.2 Optimization of Error Variance in the Encrypted Data 51 4.2.1 Tagged Information for Ciphertext 52 4.2.2 WorstCase Assumption 53 4.2.3 Error in Homomorphic Operations of the CKKS Scheme 54 4.2.4 Reordering Homomorphic Operations 59 4.3 Near-Optimal Polynomial for Modulus Reduction 66 4.3.1 Approximate Polynomial Using L2-Norm optimization 66 4.3.2 Efficient Homomorphic Evaluation of the Approximate Polynomial 70 4.4 Optimal Approximate Polynomial and Bootstrapping of the CKKS Scheme 73 4.4.1 Polynomial Basis Error and Polynomial Evaluation in the CKKS Scheme 73 4.4.2 Variance-Minimizing Polynomial Approximation 74 4.4.3 Optimal Approximate Polynomial for Bootstrapping and Magnitude of Its Coefficients 75 4.4.4 Reducing Complexity and Error Using Odd Function 79 4.4.5 Generalization of Weight Constants and Numerical Method 80 4.5 Comparison and Implementation 84 4.6 Reduction of Level Loss in Bootstrapping 89 4.7 Implementation of the Proposed Method and Performance Comparison 92 4.7.1 Error Variance Minimization 92 4.7.2 Weight Constant and Minimum Error Variance 93 4.7.3 Comparison of the Proposed MethodWith the Previous Methods 96 5 Efficient Code-Based Signature Scheme and Cryptanalysis of Code-Based Cryptosystems 104 5.1 Introduction 104 5.2 Modified ReedMuller Codes and Proposed Signature Scheme 105 5.2.1 Partial Permutation of Generator Matrix and Modified ReedMuller Codes 105 5.2.2 Decoding of Modified ReedMuller Codes 108 5.2.3 Proposed Signature Scheme 110 5.3 Security Analysis of Modified pqsigRM 111 5.3.1 Decoding One Out of Many 112 5.3.2 Security Against Key Substitution Attacks 114 5.3.3 EUFCMA Security 114 5.4 Indistinguishability of the Public Code and Signature 120 5.4.1 Modifications of Public Code 121 5.4.2 Public Code Indistinguishability 124 5.4.3 Signature Leaks 126 5.5 Parameter Selection 126 5.5.1 Parameter Sets 126 5.5.2 Statistical Analysis for Determining Number of Partial Permutations 128 5.6 Equivalence of the Prototype IKKR and the McEliece Cryptosystems 131 5.7 Cryptanalysis of the IKKR Cryptosystems 133 5.7.1 Linearity of Two Variants of IKKR Cryptosystems 133 5.7.2 The Attack Algorithm 134 5.7.3 Implementation 135 6 Conclusion 139 6.1 Privacy-Preserving Machine Learning Without Bootstrapping 139 6.2 Variance-Minimization in the CKKS Scheme 140 6.3 L2-Norm Minimization for the Bootstrapping of the CKKS Scheme 141 6.4 Modified pqsigRM: RM Code-Based Signature Scheme 142 6.5 Cryptanalysis of the IKKR Cryptosystem 143 Abstract (In Korean) 155 Acknowlegement 158Docto

Secure Outsourced Computation on Encrypted Data

Homomorphic encryption (HE) is a promising cryptographic technique that supports computations on encrypted data without requiring decryption first. This ability allows sensitive data, such as genomic, financial, or location data, to be outsourced for evaluation in a resourceful third-party such as the cloud without compromising data privacy. Basic homomorphic primitives support addition and multiplication on ciphertexts. These primitives can be utilized to represent essential computations, such as logic gates, which subsequently can support more complex functions. We propose the construction of efficient cryptographic protocols as building blocks (e.g., equality, comparison, and counting) that are commonly used in data analytics and machine learning. We explore the use of these building blocks in two privacy-preserving applications. One application leverages our secure prefix matching algorithm, which builds on top of the equality operation, to process geospatial queries on encrypted locations. The other applies our secure comparison protocol to perform conditional branching in private evaluation of decision trees. There are many outsourced computations that require joint evaluation on private data owned by multiple parties. For example, Genome-Wide Association Study (GWAS) is becoming feasible because of the recent advances of genome sequencing technology. Due to the sensitivity of genomic data, this data is encrypted using different keys possessed by different data owners. Computing on ciphertexts encrypted with multiple keys is a non-trivial task. Current solutions often require a joint key setup before any computation such as in threshold HE or incur large ciphertext size (at best, grows linearly in the number of involved keys) such as in multi-key HE. We propose a hybrid approach that combines the advantages of threshold and multi-key HE to support computations on ciphertexts encrypted with different keys while vastly reducing ciphertext size. Moreover, we propose the SparkFHE framework to support large-scale secure data analytics in the Cloud. SparkFHE integrates Apache Spark with Fully HE to support secure distributed data analytics and machine learning and make two novel contributions: (1) enabling Spark to perform efficient computation on large datasets while preserving user privacy, and (2) accelerating intensive homomorphic computation through parallelization of tasks across clusters of computing nodes. To our best knowledge, SparkFHE is the first addressing these two needs simultaneously