정보 보호 기계 학습의 암호학 기반 기술: 근사 동형 암호와 부호 기반 암호

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2021. 2. 노종선.In this dissertation, three main contributions are given as; i) a protocol of privacy-preserving machine learning using network resources, ii) the development of approximate homomorphic encryption that achieves less error and high-precision bootstrapping algorithm without compromising performance and security, iii) the cryptanalysis and the modification of code-based cryptosystems: cryptanalysis on IKKR cryptosystem and modification of the pqsigRM, a digital signature scheme proposed to the post-quantum cryptography (PQC) standardization of National Institute of Standards and Technology (NIST). The recent development of machine learning, cloud computing, and blockchain raises a new privacy problem; how can one outsource computation on confidential data? Moreover, as research on quantum computers shows success, the need for PQC is also emerging. Multi-party computation (MPC) is the cryptographic protocol that makes computation on data without revealing it. Since MPC is designed based on homomorphic encryption (HE) and PQC, research on designing efficient and safe HE and PQC is actively being conducted. First, I propose a protocol for privacy-preserving machine learning (PPML) that replaces bootstrapping of homomorphic encryption with network resources. In general, the HE ciphertext has a limited depth of circuit that can be calculated, called the level of a ciphertext. We call bootstrapping restoring the level of ciphertext that has exhausted its level through a method such as homomorphic decryption. Bootstrapping of homomorphic encryption is, in general, very expensive in time and space. However, when deep computations like deep learning are performed, it is required to do bootstrapping. In this protocol, both the client's message and servers' intermediate values are kept secure, while the client's computation and communication complexity are light. Second, I propose an improved bootstrapping algorithm for the CKKS scheme and a method to reduce the error by homomorphic operations in the CKKS scheme. The Cheon-Kim-Kim-Song (CKKS) scheme (Asiacrypt '17) is one of the highlighted fully homomorphic encryption (FHE) schemes as it is efficient to deal with encrypted real numbers, which are the usual data type for many applications such as machine learning. However, the precision drop due to the error growth is a drawback of the CKKS scheme for data processing. I propose a method to achieve high-precision approximate FHE using the following two methods .First, I apply the signal-to-noise ratio (SNR) concept and propose methods to maximize SNR by reordering homomorphic operations in the CKKS scheme. For that, the error variance is minimized instead of the upper bound of error when we deal with the encrypted data. Second, from the same perspective of minimizing error variance, I propose a new method to find the approximate polynomials for the CKKS scheme. The approximation method is especially applied to the CKKS scheme's bootstrapping, where we achieve bootstrapping with smaller error variance compared to the prior arts. In addition to the above variance-minimizing method, I cast the problem of finding an approximate polynomial for a modulus reduction into an L2-norm minimization problem. As a result, I find an approximate polynomial for the modulus reduction without using the sine function, which is the upper bound for the polynomial approximation of the modulus reduction. By using the proposed method, the constraint of q = O(m^{3/2}) is relaxed as O(m), and thus the level loss in bootstrapping can be reduced. The performance improvement by the proposed methods is verified by implementation over HE libraries, that is, HEAAN and SEAL. The implementation shows that by reordering homomorphic operations and using the proposed polynomial approximation, the reliability of the CKKS scheme is improved. Therefore, the quality of services of various applications using the proposed CKKS scheme, such as PPML, can be improved without compromising performance and security. Finally, I propose an improved code-based signature scheme and cryptanalysis of code-based cryptosystems. A novel code-based signature scheme with small parameters and an attack algorithm on recent code-based cryptosystems are presented in this dissertation. This scheme is based on a modified Reed-Muller (RM) code, which reduces the signing complexity and key size compared with existing code-based signature schemes. The proposed scheme has the advantage of the pqsigRM decoder and uses public codes that are more difficult to distinguish from random codes. I use (U, U+V) -codes with the high-dimensional hull to overcome the disadvantages of code-based schemes. The proposed a decoder which efficiently samples from coset elements with small Hamming weight for any given syndrome. The proposed signature scheme resists various known attacks on RM code-based cryptography. For 128 bits of classical security, the signature size is 4096 bits, and the public key size is less than 1 MB. Recently, Ivanov, Kabatiansky, Krouk, and Rumenko (IKKR) proposed three new variants of the McEliece cryptosystem (CBCrypto 2020, affiliated with Eurocrypt 2020). This dissertation shows that one of the IKKR cryptosystems is equal to the McEliece cryptosystem. Furthermore, a polynomial-time attack algorithm for the other two IKKR cryptosystems is proposed. The proposed attack algorithm utilizes the linearity of IKKR cryptosystems. Also, an implementation of the IKKR cryptosystems and the proposed attack is given. The proposed attack algorithm finds the plaintext within 0.2 sec, which is faster than the elapsed time for legitimate decryption.본 논문은 크게 다음의 세 가지의 기여를 포함한다. i) 네트워크를 활용해서 정보 보호 딥러닝을 개선하는 프로토콜 ii) 근사 동형 암호에서 보안성과 성능의 손해 없이 에러를 낮추고 높은 정확도로 부트스트래핑 하는 방법 iii) IKKR 암호 시스템과 pqsigRM 등 부호 기반 암호를 공격하는 방법과 효율적인 부호 기반 전자 서명 시스템. 근래의 기계학습과 블록체인 기술의 발전으로 인해서 기밀 데이터에 대한 연산을 어떻게 외주할 수 있느냐에 대한 새로운 보안 문제가 대두되고 있다. 또한, 양자 컴퓨터에 관한 연구가 성공을 거듭하면서, 이를 이용한 공격에 저항하는 포스트 양자 암호의 필요성 또한 커지고 있다. 다자간 컴퓨팅은 데이터를 공개하지 않고 데이터에 대한 연산을 수행할 수 있도록 하는 암호학적 프로토콜의 총칭이다. 다자간 컴퓨팅은 동형 암호와 포스트 양자 암호에 기반하고 있으므로, 효율적인 동형 암호와 포스트 양자 암호에 관한 연구가 활발하게 수행되고 있다. 동형 암호는 암호화된 데이터에 대한 연산이 가능한 특수한 암호화 알고리즘이다. 일반적으로 동형 암호의 암호문에 대해서 수행 가능한 연산의 깊이가 정해져 있으며, 이를 암호문의 레벨이라고 칭한다. 레벨을 모두 소비한 암호문의 레벨을 다시 복원하는 과정을 부트스트래핑 (bootstrapping)이라고 칭한다. 일반적으로 부트스트래핑은 매우 오래 걸리는 연산이며 시간 및 공간 복잡도가 크다. 그러나, 딥러닝과 같이 깊이가 큰 연산을 수행하는 경우 부트스트래핑이 필수적이다. 본 논문에서는 정보 보호 기계학습을 위한 새로운 프로토콜을 제안한다. 이 프로토콜에서는 입력 메시지와 더불어 신경망의 중간값들 또한 안전하게 보호된다. 그러나 여전히 사용자의 통신 및 연산 복잡도는 낮게 유지된다. Cheon, Kim, Kim 그리고 Song (CKKS)가 제안한 암호 시스템 (Asiacrypt 17)은 기계학습 등에서 가장 널리 쓰이는 데이터인 실수를 효율적으로 다룰 수 있으므로 가장 촉망받는 완전 동형 암호 시스템이다. 그러나, 오류의 증폭과 전파가 CKKS 암호 시스템의 가장 큰 단점이다. 이 논문에서는 아래의 기술을 활용하여 CKKS 암호 시스템의 오류를 줄이는 방법을 제안하며, 이는 근사 동형 암호에 일반화하여 적용할 수 있다. 첫째, 신호 대비 잡음 비 (signal-to-noise ratio, SNR)의 개념을 도입하여, SNR를 최대화하도록 연산의 순서를 재조정한다. 그러기 위해서는, 오류의 최대치 대신 분산이 최소화되어야 하며, 이를 관리해야 한다. 둘째, 오류의 분산을 최소화한다는 같은 관점에서 새로운 다항식 근사 방법을 제안한다. 이 근사 방법은 특히, CKKS 암호 시스템의 부트스트래핑에 적용되었으며, 종래 기술보다 더 낮은 오류를 달성한다. 위의 방법에 더하여, 근사 다항식을 구하는 문제를 L2-norm 최소화 문제로 치환하는 방법을 제안한다. 이를 통해서 사인 함수의 도입 없이 근사 다항식을 구하는 방법을 제안한다. 제안된 방법을 사용하면, q=O(m^{3/2})라는 제약을 q=O(m)으로 줄일 수 있으며, 부트스트래핑에 필요한 레벨 소모를 줄일 수 있다. 성능 향상은 HEAAN과 SEAL 등의 동형 암호 라이브러리를 활용한 구현을 통해 증명했으며, 구현을 통해서 연산 재정렬과 새로운 부트스트래핑이 CKKS 암호 시스템의 성능을 향상함을 확인했다. 따라서, 보안성과 성능의 타협 없이 근사 동형 암호를 사용하는 서비스의 질을 향상할 수 있다. 양자 컴퓨터를 활용하여 전통적인 공개키 암호를 공격하는 효율적인 알고리즘이 공개되면서, 포스트 양자 암호에 대한 필요성이 증대했다. 부호 기반 암호는 포스트 양자 암호로써 널리 연구되었다. 작은 키 크기를 갖는 새로운 부호 기반 전자 서명 시스템과 부호 기반 암호를 공격하는 방법이 논문에 제안되어 있다. pqsigRM이라 명명한 전자 서명 시스템이 그것이다. 이 전자 서명 시스템은 수정된 Reed-Muller (RM) 부호를 활용하며, 서명의 복잡도와 키 크기를 종래 기술보다 많이 줄인다. pqsigRM은 hull의 차원이 큰 (U, U+V) 부호와 이의 복호화를 이용하여, 서명에서 큰 이득이 있다. 이 복호화 알고리즘은 주어진 모든 코셋 (coset)의 원소에 대하여 작은 헤밍 무게를 갖는 원소를 반환한다. 또한, 수정된 RM 부호를 이용하여, 알려진 모든 공격에 저항한다. 128비트 안정성에 대해서 서명의 크기는 4096 비트이고, 공개 키의 크기는 1MB보다 작다. 최근, Ivanov, Kabatiansky, Krouk, 그리고 Rumenko (IKKR)가 McEliece 암호 시스템의 세 가지 변형을 발표했다 (CBCrypto 2020, Eurocrypt 2020와 함께 진행). 본 논문에서는 IKKR 암호 시스템중 하나가 McEliece 암호 시스템과 동치임을 증명한다. 또한 나머지 IKKR 암호 시스템에 대한 다항 시간 공격을 제안한다. 제안하는 공격은 IKKR 암호 시스템의 선형성을 활용한다. 또한, 이 논문은 제안한 공격의 구현을 포함하며, 제안된 공격은 0.2초 이내에 메시지를 복원하고, 이는 정상적인 복호화보다 빠른 속도이다.Contents Abstract i Contents iv List of Tables ix List of Figures xi 1 Introduction 1 1.1 Homomorphic Encryption and Privacy-Preserving Machine Learning 4 1.2 High-Precision CKKS Scheme and Its Bootstrapping 5 1.2.1 Near-Optimal Bootstrapping of the CKKS Scheme Using Least Squares Method 6 1.2.2 Variance-Minimizing and Optimal Bootstrapping of the CKKS Scheme 8 1.3 Efficient Code-Based Signature Scheme and Cryptanalysis of the Ivanov-Kabatiansky-Krouk-Rumenko Cryptosystems 10 1.3.1 Modified pqsigRM: An Efficient Code-Based Signature Scheme 11 1.3.2 Ivanov-Kabatiansky-Krouk-Rumenko Cryptosystems and Its Equality 13 1.4 Organization of the Dissertation 14 2 Preliminaries 15 2.1 Basic Notation 15 2.2 Privacy-Preserving Machine Learning and Security Terms 16 2.2.1 Privacy-Preserving Machine Learning and Security Terms 16 2.2.2 Privacy-Preserving Machine Learning 17 2.3 The CKKS Scheme and Its Bootstrapping 18 2.3.1 The CKKS Scheme 18 2.3.2 CKKS Scheme in RNS 22 2.3.3 Bootstrapping of the CKKS Scheme 24 2.3.4 Statistical Characteristics of Modulus Reduction and Failure Probability of Bootstrapping of the CKKS Scheme 26 2.4 Approximate Polynomial and Signal-to-Noise Perspective for Approximate Homomorphic Encryption 27 2.4.1 Chebyshev Polynomials 27 2.4.2 Signal-to-Noise Perspective of the CKKS Scheme 28 2.5 Preliminary for Code-Based Cryptography 29 2.5.1 The McEliece Cryptosystem 29 2.5.2 CFS Signature Scheme 30 2.5.3 ReedMuller Codes and Recursive Decoding 31 2.5.4 IKKR Cryptosystems 33 3 Privacy-Preserving Machine Learning via FHEWithout Bootstrapping 37 3.1 Introduction 37 3.2 Information Theoretic Secrecy and HE for Privacy-Preserving Machine Learning 38 3.2.1 The Failure Probability of Ordinary CKKS Bootstrapping 39 3.3 Comparison With Existing Methods 43 3.3.1 Comparison With the Hybrid Method 43 3.3.2 Comparison With FHE Method 44 3.4 Comparison for Evaluating Neural Network 45 4 High-Precision Approximate Homomorphic Encryption and Its Bootstrapping by Error Variance Minimization and Convex Optimization 50 4.1 Introduction 50 4.2 Optimization of Error Variance in the Encrypted Data 51 4.2.1 Tagged Information for Ciphertext 52 4.2.2 WorstCase Assumption 53 4.2.3 Error in Homomorphic Operations of the CKKS Scheme 54 4.2.4 Reordering Homomorphic Operations 59 4.3 Near-Optimal Polynomial for Modulus Reduction 66 4.3.1 Approximate Polynomial Using L2-Norm optimization 66 4.3.2 Efficient Homomorphic Evaluation of the Approximate Polynomial 70 4.4 Optimal Approximate Polynomial and Bootstrapping of the CKKS Scheme 73 4.4.1 Polynomial Basis Error and Polynomial Evaluation in the CKKS Scheme 73 4.4.2 Variance-Minimizing Polynomial Approximation 74 4.4.3 Optimal Approximate Polynomial for Bootstrapping and Magnitude of Its Coefficients 75 4.4.4 Reducing Complexity and Error Using Odd Function 79 4.4.5 Generalization of Weight Constants and Numerical Method 80 4.5 Comparison and Implementation 84 4.6 Reduction of Level Loss in Bootstrapping 89 4.7 Implementation of the Proposed Method and Performance Comparison 92 4.7.1 Error Variance Minimization 92 4.7.2 Weight Constant and Minimum Error Variance 93 4.7.3 Comparison of the Proposed MethodWith the Previous Methods 96 5 Efficient Code-Based Signature Scheme and Cryptanalysis of Code-Based Cryptosystems 104 5.1 Introduction 104 5.2 Modified ReedMuller Codes and Proposed Signature Scheme 105 5.2.1 Partial Permutation of Generator Matrix and Modified ReedMuller Codes 105 5.2.2 Decoding of Modified ReedMuller Codes 108 5.2.3 Proposed Signature Scheme 110 5.3 Security Analysis of Modified pqsigRM 111 5.3.1 Decoding One Out of Many 112 5.3.2 Security Against Key Substitution Attacks 114 5.3.3 EUFCMA Security 114 5.4 Indistinguishability of the Public Code and Signature 120 5.4.1 Modifications of Public Code 121 5.4.2 Public Code Indistinguishability 124 5.4.3 Signature Leaks 126 5.5 Parameter Selection 126 5.5.1 Parameter Sets 126 5.5.2 Statistical Analysis for Determining Number of Partial Permutations 128 5.6 Equivalence of the Prototype IKKR and the McEliece Cryptosystems 131 5.7 Cryptanalysis of the IKKR Cryptosystems 133 5.7.1 Linearity of Two Variants of IKKR Cryptosystems 133 5.7.2 The Attack Algorithm 134 5.7.3 Implementation 135 6 Conclusion 139 6.1 Privacy-Preserving Machine Learning Without Bootstrapping 139 6.2 Variance-Minimization in the CKKS Scheme 140 6.3 L2-Norm Minimization for the Bootstrapping of the CKKS Scheme 141 6.4 Modified pqsigRM: RM Code-Based Signature Scheme 142 6.5 Cryptanalysis of the IKKR Cryptosystem 143 Abstract (In Korean) 155 Acknowlegement 158Docto

"Once Upon a Place": Compute Your Meeting Location Privately

Popular services such as Doodle Mobile and Tymelie are extremely useful planning tools that enable mobile-phone users to determine common meeting time(s) for events. Similar planning tools for determining optimal meeting locations, based on the location preferences of the users, are highly desirable for event planning and management in popular mobile phone applications, such as taxi sharing, route planning and mobile participatory sensing. Yet, they have received very little attention by researchers. An important, and often overlooked, facet of such planning applications is the privacy of the participating users and their preferences; users want to agree on a meeting location without necessarily revealing their location preferences to the service provider or to the other users. In this paper, we address the problem of privacy-preserving optimal meeting-location computation, especially focusing on its applicability to current mobile devices and applications. We first define the notion of privacy in such computations. Second, we model the problem of optimal meeting-location computation as a privacy-preserving k-center problem and we design two solutions; both solutions take advantage of the homomorphic properties of well-known cryptosystems by Boneh-Goh-Nissim, ElGamal and Paillier in order to perform oblivious computations. Third, we implement the proposed solutions on a testbed of the latest generation Nokia mobile devices and study their performance. Finally, we assess the utility and expectations, in terms of privacy and usability, of the proposed solutions by means of a targeted survey and user-study of mobile-phone users

DISTRIBUTED LEARNING ALGORITHMS: COMMUNICATION EFFICIENCY AND ERROR RESILIENCE

In modern day machine learning applications such as self-driving cars, recommender systems, robotics, genetics etc., the size of the training data has grown to the point that it has become essential to design distributed learning algorithms. A general framework for the distributed learning is \emph{data parallelism} where the data is distributed among the \emph{worker machines} for parallel processing and computation to speed up learning. With billions of devices such as cellphones, computers etc., the data is inherently distributed and stored locally in the users\u27 devices. Learning in this set up is popularly known as \emph{Federated Learning}. The speed-up due to distributed framework gets hindered by some fundamental problems such as straggler workers, communication bottleneck due to high communication overhead between workers and central server, adversarial failure popularly know as \emph{Byzantine failure}. In this thesis, we study and develop distributed algorithms that are error resilient and communication efficient. First, we address the problem of straggler workers where the learning is delayed due to slow workers in the distributed setup. To mitigate the effect of the stragglers, we employ \textbf{LDPC} (low density parity check) code to encode the data and implement gradient descent algorithm in the distributed setup. Second, we present a family of vector quantization schemes \emph{vqSGD} (vector quantized Stochastic Gradient Descent ) that provides an asymptotic reduction in the communication cost with convergence guarantees in the first order distributed optimization. We also showed that \emph{vqSGD} provides strong privacy guarantee. Third, we address the problem of Byzantine failure together with communication-efficiency in the first order gradient descent algorithm. We consider a generic class of $\delta$- approximate compressor for communication efficiency and employ a simple \emph{norm based thresholding} scheme to make the learning algorithm robust to Byzantine failures. We establish statistical error rate for non-convex smooth loss. Moreover, we analyze the compressed gradient descent algorithm with error feedback in a distributed setting and in the presence of Byzantine worker machines. Fourth, we employ the generic class of $\delta$- approximate compressor to develop a communication efficient second order Newton-type algorithm and provide rate of convergence for smooth objective. Fifth, we propose \textbf{COMRADE} (COMmunication-efficient and Robust Approximate Distributed nEwton ), an iterative second order algorithm that is communication efficient as well as robust against Byzantine failures. Sixth, we propose a distributed \emph{cubic-regularized Newton } algorithm that can escape saddle points effectively for non-convex loss function and find a local minima . Furthermore, the proposed algorithm can resist the attack of the Byzantine machines, which may create \emph{fake local minima} near the saddle points of the loss function, also known as saddle-point attack

Set-valued Data: Regression, Design and Outliers

The focus of this dissertation is to study set‐valued data from three aspects, namely regression, optimal design and outlier identification. This dissertation consists of three peer‐reviewed published articles, each of them addressing one aspect. Their titles and abstracts are listed below: 1. Local regression smoothers with set‐valued outcome data: This paper proposes a method to conduct local linear regression smoothing in the presence of set‐valued outcome data. The proposed estimator is shown to be consistent, and its mean squared error and asymptotic distribution are derived. A method to build error tubes around the estimator is provided, and a small Monte Carlo exercise is conducted to confirm the good finite sample properties of the estimator. The usefulness of the method is illustrated on a novel dataset from a clinical trial to assess the effect of certain genes’ expressions on different lung cancer treatments outcomes. 2. Optimal design for multivariate multiple linear regression with set‐identified response: We consider the partially identified regression model with set‐identified responses, where the estimator is the set of the least square estimators obtained for all possible choices of points sampled from set‐identified observations. We address the issue of determining the optimal design for this case and show that, for objective functions mimicking those for several classical optimal designs, their set‐identified analogues coincide with the optimal designs for point‐identified real‐valued responses. 3. Depth and outliers for samples of sets and random sets distributions: We suggest several constructions suitable to define the depth of set‐valued observations with respect to a sample of convex sets or with respect to the distribution of a random closed convex set. With the concept of a depth, it is possible to determine if a given convex set should be regarded an outlier with respect to a sample of convex closed sets. Some of our constructions are motivated by the known concepts of half‐space depth and band depth for function‐valued data. A novel construction derives the depth from a family of non‐linear expectations of random sets. Furthermore, we address the role of positions of sets for evaluation of their depth. Two case studies concern interval regression for Greek wine data and detection of outliers in a sample of particles

On constructions of quantum-secure device-independent randomness expansion protocols

Device-independent randomness expansion protocols aim to expand a short uniformly random string into a much longer one whilst guaranteeing that their output is truly random. They are device-independent in the sense that this guarantee does not dependent on the specifics of an implementation. Rather, through the observation of nonlocal correlations we can conclude that the outputs generated are necessarily random. This thesis reports a general method for constructing these protocols and evaluating their security. Using this method, we then construct several explicit protocols and analyse their performance on noisy qubit systems. With a view towards near-future quantum technologies, we also investigate whether randomness expansion is possible using current nonlocality experiments. We find that, by combining the recent theoretical and experimental advances, it is indeed now possible to reliably and securely expand randomness

A Survey on Quantum Channel Capacities

Quantum information processing exploits the quantum nature of information. It offers fundamentally new solutions in the field of computer science and extends the possibilities to a level that cannot be imagined in classical communication systems. For quantum communication channels, many new capacity definitions were developed in comparison to classical counterparts. A quantum channel can be used to realize classical information transmission or to deliver quantum information, such as quantum entanglement. Here we review the properties of the quantum communication channel, the various capacity measures and the fundamental differences between the classical and quantum channels.Comment: 58 pages, Journal-ref: IEEE Communications Surveys and Tutorials (2018) (updated & improved version of arXiv:1208.1270