Converting to Optimization in Machine Learning: Perturb-and-MAP, Differential Privacy, and Program Synthesis

On a mathematical level, most computational problems encountered in machine learning are instances of one of four abstract, fundamental problems: sampling, integration, optimization, and search. Thanks to the rich history of the respective mathematical fields, disparate methods with different properties have been developed for these four problem classes. As a result it can be beneficial to convert a problem from one abstract class into a problem of a different class, because the latter might come with insights, techniques, and algorithms well suited to the particular problem at hand. In particular, this thesis contributes four new methods and generalizations of existing methods for converting specific non-optimization machine learning tasks into optimization problems with more appealing properties. The first example is partition function estimation (an integration problem), where an existing algorithm -- the Gumbel trick -- for converting to the MAP optimization problem is generalized into a more general family of algorithms, such that other instances of this family have better statistical properties. Second, this family of algorithms is further generalized to another integration problem, the problem of estimating Rényi entropies. The third example shows how an intractable sampling problem arising when wishing to publicly release a database containing sensitive data in a safe ("differentially private") manner can be converted into an optimization problem using the theory of Reproducing Kernel Hilbert Spaces. Finally, the fourth case study casts the challenging discrete search problem of program synthesis from input-output examples as a supervised learning task that can be efficiently tackled using gradient-based optimization. In all four instances, the conversions result in novel algorithms with desirable properties. In the first instance, new generalizations of the Gumbel trick can be used to construct statistical estimators of the partition function that achieve the same estimation error while using up to 40% fewer samples. The second instance shows that unbiased estimators of the Rényi entropy can be constructed in the Perturb-and-MAP framework. The main contribution of the third instance is theoretical: the conversion shows that it is possible to construct an algorithm for releasing synthetic databases that approximate databases containing sensitive data in a mathematically precise sense, and to prove results about their approximation errors. Finally, the fourth conversion yields an algorithm for synthesising program source code from input-output examples that is able to solve test problems 1-3 orders of magnitude faster than a wide range of baselines

Public keys quality

Dissertação de mestrado em Matemática e ComputaçãoThe RSA cryptosystem, invented by Ron Rivest, Adi Shamir and Len Adleman ([Rivest et al., 1978]) is the most commonly used cryptosystem for providing privacy and ensuring authenticity of digital data. RSA is usually used in contexts where security of digital data is priority. RSA is used worldwide by web servers and browsers to secure web traffic, to ensure privacy and authenticity of e-mail, to secure remote login sessions and to provide secure electronic creditcard payment systems. Given its importance in the protection of digital data, vulnerabilities of RSA have been analysed by many researchers. The researches made so far led to a number of fascinating attacks. Although the attacks helped to improve the security of this cryptosystem, showing that securely implementing RSA is a nontrivial task, none of them was devastating. This master thesis discusses the RSA cryptosystem and some of its vulnerabilities as well as the description of some attacks, both recent and old, together with the description of the underlying mathematical tools they use. Although many types of attacks exist, in this master thesis only a few examples were analysed. The ultimate attack, based in the batch-GCD algorithm, was implemented and tested in the RSA keys produced by a certificated Hardware Security Modules Luna SA and the results were commented. The random and pseudorandom numbers are fundamental to many cryptographic applications, including the RSA cryptosystems. In fact, the produced keys must be generated in a specific random way. The National Institute of Standards and Technology, responsible entity for specifying safety standards, provides a package named "A Statistical Test Suit for Random and Pseudorandom Number Generators for Cryptography Applications" which was used in this work to test the randomness of the Luna SA generated numbers. All the statistical tests were tested in different bit sizes number and the results commented. The main purpose of this thesis is to study the previous subjects and create an applications capable to test the Luna SA generated numbers randomness, a well as evaluate the security of the RSA. This work was developed in partnership with University of Minho and Multicert.O RSA, criado por Ron Rivest, Adi Shamir e Len Adleman ([Rivest et al., 1978]) é o sistema criptográfico mais utilizado para providenciar segurança e assegurar a autenticação de dados utilizados no mundo digital. O RSA é usualmente usado em contextos onde a segurança é a grande prioridade. Hoje em dia, este sistema criptográfico é utilizado mundialmente por servidores web e por browsers, por forma a assegurar um tráfego seguro através da Internet. É o sistema criptográfico mais utilizado na autenticação de e-mails, nos inícios de sessões remotos, na utilização de pagamentos através de cartões multibanco, garantindo segurança na utilização destes serviços. Dada a importância que este sistema assume na proteção da informação digital, as suas vulnerabilidades têm sido alvo de várias investigações. Estas investigações resultaram em vários ataques ao RSA. Embora nenhum destes ataques seja efetivamente eficaz, todos contribuíram para um aumento da segurança do RSA, uma vez que as implementações de referência deste algoritmo passaram a precaver-se contra os ataques descobertos. Esta tese de mestrado aborda o sistema criptográfico RSA, discutindo algumas das suas vulnerabilidades, assim como alguns ataques efetuados a este sistema, estudando todos os métodos matemáticos por estes usados. Embora existam diversos ataques, apenas alguns serão abordados nesta tese de mestrado. O último ataque, baseado no algoritmo batch-GCD foi implementado e foram feitos testes em chaves RSA produzidas por um Hardware Security Module Luna SA certificado e os resultados obtidos foram discutidos. Os números aleatórios e pseudoaleatórios são fundamentais a todas as aplicações criptográficas, incluindo, portanto, o sistema criptográfico RSA. De facto, as chaves produzidas deverão ser geradas com alguma aleatoriedade intrínseca ao sistema. O Instituto Nacional de Standards e Tecnologia, entidade responsável pela especificação dos standards de segurança, disponibiliza um pacote de testes estatísticos, denominado por "A Statistical Test Suit for Random and Pseudorandom Number Generators for Cryptography Applications". Estes testes estatísticos foram aplicados a números gerados pelo Luna SA e os resultados foram, também, comentados. O objetivo desta tese de mestrado é desenvolver capacidade de compreensão sobre os assuntos descritos anteriormente e criar uma aplicação capaz de testar a aleatoriedade dos números gerados pelo Luna SA, assim como avaliar a segurança do sistema criptográfico RSA. Este foi um trabalho desenvolvido em parceria com a Universidade do Minho e com a Multicert

Computer Simulations in Science and Engineering. Concept, Practices, Perspectives

This book addresses key conceptual issues relating to the modern scientific and engineering use of computer simulations. It analyses a broad set of questions, from the nature of computer simulations to their epistemological power, including the many scientific, social and ethics implications of using computer simulations. The book is written in an easily accessible narrative, one that weaves together philosophical questions and scientific technicalities. It will thus appeal equally to all academic scientists, engineers, and researchers in industry interested in questions related to the general practice of computer simulations

Molecular simulations in zeolites for industrial applications

Programa de Doctorado en Biotecnología, Ingeniería y Tecnología QuímicaLínea de Investigación: Experimentación y Computación en Materiales y Sistemas ComplejosClave Programa: DBICódigo Línea: 15Zeolites are known for their suitability as molecular sieves. Actually, they are already firmly settled in industry to selectively separate gases or capture them. In this thesis, the processes selected share the objective of contributing to make more competitive or enhance alternative processes to those based on carbon technology. These processes have been approached from a computational point of view. Molecular simulations have been carried out to gain molecular insight on gas mixtures separation processes at the same time as obtaining valuable information to suggest operational schemes to perform these separations. Additionally, some specific underlying molecular phenomena have been tackled to study supplementary aspects, not centered on particular industrial processes but relevant to understand the behavior of adsorbed molecules in separation procedures. Therefore, two blocks can be differentiated in this thesis: Gas mixture separations for industrial processes Chapter 2 An operating procedure is proposed to separate the components of a tail gas expelled from a Fischer-Tropsch process. The quinary gas mixture is composed of light gases, i.e. CO2 , CO, CH4 , N2 , and H2 in a typical, described composition. The Fischer-Tropsch process is a key step in a global gas-to-liquid process to synthesize hydrocarbons. The separation procedure has a twofold goal: upstream reintroduction of the captured CH4 and CO and trapping of CO2 to avoid its emission. To that end, the performance in the described separation is assessed for four zeolites of high industrial impact (DDR, FAU, MFI, and MOR), with main focus on the effect of location and amount of aluminum atoms in the zeolite lattices. The decision on the final scheme is based on the combination of computed adsorption isotherms, selectivity and diffusion coefficients. In addition, applicability and accuracy of IAST is studied in these adsorbate¿adsobent systems. Chapter 3 A large multi-scale computational study is performed to achieve a selective separation of a mixture of CO2 , CO, and O2 gases. The separation takes place in the context of the nonthermal plasma-assisted CO2 dissociation process, involved in a process scheme aiming at the production of fuels with neutral carbon footprint. The proposed route requires an additional step to obtain pure CO from the mixture and avoid a high CO2 recombination. A widespread screening on 174 zeolites, evaluating selectivity at low coverage and successive adsorption simulations at molecular level for specific structures, combined with IAST, provides a scheme to perform the separation under mild operation conditions. Then PSA simulations are carried out to find the optimal parameters to achieve the desired separation at a pilot-plant scale. Chapter 4 Deuterium and tritium from hydrogen separations are studied over a wide range of pressures and low and cryogenic temperatures. Due to the nature of the adsorbates and operation conditions, quantum corrections are considered. Models for the deuterium and tritium molecules are proposed, derived from the hydrogen model previously reported. Experimental adsorption isotherms for H2 and D2 respectively check and validate the models for such molecules in pure silica zeolites. Then, a study on the adsorption selectivity at infinite dilution is done on 210 pure silica zeolites, and subsequent diffusion and adsorption simulations are performed over a range of pressures and temperatures for the most promising zeolites. Three zeolites, BCT, AVL, and MVY, are identified as the best candidates to perform a separation of a 1:1 D2/H2 mixture. One of them, BCT topology, is found to show, at low temperature, the highest adsorption selectivity reported to the best of our knowledge. The same structure is also found to obtain an extremely high selectivity for a 1:1 T2/H2 mixture. Molecular insights on additional aspects for molecular separation Chapter 5 A study on the effect of cations on diffusion of CO2 and CH4 molecules in MFI zeolite is carried out. The industrial relevance of both molecules and of the zeolite framework is beyond all discussion. Theoretical aluminum distributions are generated considering the 12 T crystallographic positions of framework MFI. Monovalent and divalent counterbalancing cations are considered to neutralize the negative charge introduced in the system by the aluminum atoms. Probability density of cations and energy profiles for the adsorbates, both depending on aluminum distributions, are evaluated together to produce a prediction on the behavior of adsorbates and cations, which is found to be consistent with subsequent diffusion simulations. All the results shed light on why zeolites with the same chemical composition have different dynamical behaviors. Chapter 6 Understanding the role of water in LTA zeolite is key due to the use of this topology in water removal and dehydration processes. The wide range of aluminum substitutions available experimentally for this topology, from the pure siliceous lattice up to the theoretical maximum of Si:Al=1, means the degree of hydrophobicity or hydrophilicity of the zeolite can be tailored. In the theoretical study, besides considering different Si:Al ratios, two lattices are examined for each one, keeping the crystallographic positions fixed after substitutions and allowing the lattice to relax. Adsorption isotherms in fixed and energy-minimized lattices and a thorough analysis of the location of water molecules reveals that: adsorption sites are determined by the hydrophilicity of the lattice. The more hydrophilic, the bigger the lattice, which reinforces the ability of the structure to adsorb in the narrow pores. The pressure is found to affect strongly the preference for large or narrow pores. A case is identified, in which at increasing pressures or loading narrow pores are first populated, then emptied as the large pores fill, and then finally populated again. Chapter7 ConclusionsUniversidad Pablo de Olavide de Sevilla. Departamento de Sistemas Físicos, Químicos y NaturalesPostprin

Assessment of Reactivity Descriptors for the Study of Chemical Reactions

At present, a plethora of techniques is available to study molecular systems and chemical reactions. In this light, ab initio many body techniques enable us, without any experimental input, to compute molecular energetics and a whole range of chemical properties. A twofold classification can be made, separating wave function based methods (such as Hartree-Fock) from density functional theory (DFT). In this thesis, the focus mainly lies on the density functional method, both from a conceptual as computational viewpoint - for the latter, one of the reasons being its excellent cost-to-performance ratio. The formulation of the Hohenberg-Kohn theorems in 1964 provided the starting point for the fast development of fundamental DFT, yielding all basic ingredients for a complete many-body theory. In a next step, Kohn and Sham introduced orbitals within the formalism, paving the way to a breakthrough of DFT as an important computational tool within modern chemistry. In addition to these developments, much attention has been paid to the insights DFT can give into chemical reactivity. In particular, the density functional methodology provides an excellent framework to define a variety of well-known chemical concepts. In this thesis, the applicability of several fundamental DFT-based reactivity indicators has been tested for a variety of reactions occurring in diverse domains of chemistry. The chemical potential, global hardness and global softness are used to discuss the reactive behavior of one single molecule or a set of related molecules, providing reactivity sequences for the latter. The Fukui function is used as an intra-molecular descriptor, whereas the local softness can provide inter-molecular reactivity information about regio selectivity. In order to test the usefulness of the aforementioned properties, they are usually used in conjunction with well-known chemical principles such as the electronegativity equalization, the principle of maximum hardness and the hard and soft acids and bases (HSAB) principle. A validation of the results is obtained through comparison with ab initio thermodynamic and kinetic data. In general, DFT-based concepts have proven to be of great use in the interpretation of a variety of experimental and theoretical results. It remains nevertheless challenging to make predictions about their success or failure for specific types of interactions. In order to rely on the indicators as independent and/or predictive tools for the description of chemical reactivity, there is need for practical guidelines describing which indicators are suitable for a certain problem. The descriptors probe electronic interactions, however it is clear that they do not monitor other effects, such as steric contributions. Whenever the indicators are tested, their limitations must a priori be kept in mind. For instance, some of the reactions studied in this work exhibit high reaction barriers due to large steric hindrance effects between the two reactants and thus, the indicators are not expected to be successful. The largest group of applications concerns radical reactions. The majority of them occur during the formation of coke, which is a side process of thermal cracking of hydrocarbons. Radical systems are particularly interesting and challenging. However so far only a handful of papers examined and discussed their behavior using reactivity descriptors. In the specific case of hydrogen abstraction reactions at polyaromatic species by a methyl radical, accurate, reliable and yet affordable thermodynamic and kinetic data have been obtained. Other studied bimolecular radical reactions are addition reactions of radicals to small olefins and the initiation and first propagation steps within the polymerization process of polyethylene and poly(vinylchloride). Overall, it can be concluded that the radical Fukui function is capable of indicating the preferred site of reaction. This clearly indicates the importance of frontier orbital effects for this specific type of interaction. The local HSAB principle was also successfully applied, leading to correct reactivity sequences when related molecules were compared with each other. Hence the local softness is a suitable indicator for studying bimolecular radical reactions. This is in accordance with earlier suggestions and with the categorization of Klopman, as the larger radicals are characterized by high global softness values. Whereas the local viewpoint leads to satisfactory results, the global HSAB principle fails for some specific bimolecular radical reactions and is therefore not recommended. In addition to these bimolecular reactions, two examples of unimolecular radical cyclization reactions have also been studied. For these reactions, the Fukui function is used to describe the intra-molecular regio selectivity. However, the correct site of cyclization could not be indicated. Since these reactions correspond to a constrained charge transfer - the spin number remains constant, whereas the total number of electrons changes at a local level - the spin-polarized reactivity descriptors, and in particular the spin-polarized Fukui function has been tested and indeed, its usefulness is demonstrated. A second group of studied bimolecular reactions correspond to typical reactions occurring within zeolite catalysis. The small probe molecules are characterized as hard systems, and both polar and apolar representatives were chosen. The zeolite catalyst was modelled by a small 5-tetrahedral cluster, and in addition to the traditional oxygen-bridged cluster the influence of amine substitution was tested. These reactions were difficult to describe using DFT-based reactivity descriptors. The global HSAB principle fails for the interactions with the polar molecules, as in these cases polarization effects - which are not included in the purely electron transfer process - are dominant. In contrast, the reactions between the zeolite clusters and the apolar molecules do follow the HSAB rule. From a local point of view, the complex mechanism of these concerted reactions, occurring simultaneously at the acid and basic site of the zeolite framework, could not be specified based on local descriptors. Multiple-site interactions in general remain a domain for further research. In addition, the local hardness might be more suitable to describe interactions with hard molecules, although at present the calculations remain troublesome and no practical scheme is presented yet. A final application concerns a unimolecular ionic cyclization reaction leading to the formation of a four-membered beta-lactam ring. This is an example in which competing pathways can take place because the formation of the more stable six-membered ring, instead of the strained four-membered ring, is initially expected. The anion reactant is a very soft species and the traditional local indicator, i. e. the Fukui function, succeeds in correctly predicting the preferred reaction path. Therefore the indicator adds to the unravelling of the origin of the experimentally observed cyclization preference. Moreover, extension to bimolecular reactions using this anion reactant also revealed the successful applicability of the local softness, as also these results were in correspondence with reaction barriers obtained at 0 Kelvin. In this thesis, one of the main advantages of using reactivity descriptors was accentuated, in particular their low computational cost. Not only do they solely require information from the reactants, but it was furthermore demonstrated that reactivity sequences are mainly independent of the level of theory used for geometry optimizations as well as for single-point energy calculations. An extended set of theoretical procedures, ranging from simple ab initio methods such as Hartree-Fock, over a very popular and successful functional, to the latest class of highly promising methods. The observation of level-of-theory independency for the calculation of the global descriptors is highly reassuring and adds to the reliability of DFT-based reactivity indicators. It must however be emphasized that overall the perturbative approach should be used complementary to information obtained from thermodynamics and kinetics

1996-1997 Bulletin

Volume 107, Number 4. Scanned from the copy held in the Registrar\u27s Office.https://ecommons.udayton.edu/bulletin/1045/thumbnail.jp