Algahtani, Eyad

Erten, Can

Fairlamb, Ian

Garcia-Padilla, Eduardo

Kazakov, Dimitar

Lynam, Jason

Manandhar, Suresh

Slattery, John

English

White Rose Research Online

This is a repository copy of Unsupervised Learning of Functional Groups for Computational Chemistry.White Rose Research Online URL for this paper:http://eprints.whiterose.ac.uk/157651/Conference or Workshop Item:Erten, Can, Algahtani, Eyad, Fairlamb, Ian orcid.org/0000-0002-7555-2761 et al. (5 more authors) (2019) Unsupervised Learning of Functional Groups for Computational Chemistry.In: The 29th ILP conference, 03-05 Sep 2019, Plovdiv, Bulgaria. eprints@whiterose.ac.ukhttps://eprints.whiterose.ac.uk/Reuse Items deposited in White Rose Research Online are protected by copyright, with all rights reserved unless indicated otherwise. They may be downloaded and/or printed for private study, or other acts as permitted by national copyright laws. The publisher or other rights holders may allow further reproduction and re-use of the full text version. This is indicated by the licence information on the White Rose Research Online record for the item. Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. The 29th ILP conference, 3–5 Sep 2019, Plovdiv, Bulgaria.Unsupervised Learning of Functional Groupsfor Computational ChemistryCan Erten, Eyad Algahtani, Ian Fairlamb, Eduardo Garcia-Padilla, JasonLynam, Suresh Manandhar, John Slattery, and Dimitar KazakovUniversity of York, York, UKdimitar.kazakov@york.ac.ukhttp://www.cs.york.ac.uk/∼kazakovKeywords: Unsupervised learning · Inductive Logic Programming · Featuregeneration · Concurrent computation · Computational Chemistry.1 IntroductionComputer modelling now forms a standard part of chemists’ armoury. Whilesome models are based on a combination of precise measurements and quantumtheory, others try to link what knowledge there is available about a given chem-ical compound with its observed properties through Machine Learning (ML)models.Composition and structure are always the main aspects reflected in the train-ing data, but ML approaches differ in the use of additional features signallingthe presence of certain substructures that have been deemed of relevance. At oneend, connectionist approaches of the ‘Deep Learning’ (DL) kind rely on multiplelayers of neurons to combine the original aspects of the representation throughnon-linear functions into features of increasingly higher level of complexity andabstraction; at the other extreme, more traditional statistical ML approachesrely on the explicit provision of such features identified at an earlier stage, andadded to a given data set through pre-processing.The amount of feature construction effort saved through DL is most obviouswhen the native representation of the data maps easily onto the input layer ofthe neural network: image processing is one such class of applications. Whenpotentially complex structure needs to be represented, as in Chemistry, labelsidentifying additional structural features are also explicitly added to the data.This extra effort comes on top of DL’s inherent downside of needing large trainingdata sets.Relational, logic-based representations, such as the ones used in InductiveLogic Programming (ILP) have proven very well suited to chemistry data. Theirclaimed advantage is model transparency and the ability to learn from a limitednumber of training examples, which are annotated as a rule.Here we are interested in what can be achieved through learning from unan-notated data, with a focus on identifying functional groups, as such parts of amolecule could be used as features to assist further machine learning tasks. For12 The 29th ILP conference, 3–5 Sep 2019, Plovdiv, Bulgaria.this purpose, we return to our previous work on the learning of word morphol-ogy, which showed how general learning biases related to Occam’s Razor andMinimum Description Length can be used to identify functional components inunannotated lists of words. These functional components can then be used tolabel the training data and present it to an ILP algorithm [1, 2]. The result ofsuch hybrid approach is twofold: a list of morpheme-like components which havethe ability to recombine into new words, and a set of rules that split a givenword into such functional constituents.The bias guiding the unsupervised learning step is based on the definition ofa morpheme as a functional constituent that appears next to another morphemein at least two different words. While this definition appears circular, there is away to square that particular circle using only four words in the process, as firstshown by Saussure [3]:work + s ∼ work + ingsleep+ s ∼ sleep+ ing (1)In this so-called analogy square (‘works’ is to ‘working’ what ‘sleeps’ is to ‘sleep-ing’), all four constituents satisfy the above definition when considered simulta-neously. Here we adopt a similar approach to the definition of functional group,and look for 4-tuples of molecules forming the same pattern, e.g.:Na+OH ∼ Na+ ClK+OH ∼ K+ Cl (2)Here the operator + represents a valid split for the given representation, e.g.any subset of atoms and its complement if only the classical chemical formula isconsidered or the result of a valence bond removal when this breaks the molecularstructure into two constituents.Given a list of molecules, one can select one, e.g. NaOH, and look for all setsof 3 molecules that complete an analogy square of the type shown above. Wethen: (a) assume that each split supported by at least one such square resultsin two valid functional groups, and, (b) note the split supported by the largestnumber of squares as the most likely one.2 Sources and Representation of DataThere are several public databases containing chemical data using various repre-sentations. The two representations we have been most interested in are SMILES,and MOL3D. The former is a lossy representation and does not fully describe thethree-dimensional structure of the molecule. The latter allows one to reconstructall bonds between atoms, and this is the one we focus on, using the ChemSpiderdatabase.A late breaking abstract published under a CC BY licence. 33 The MethodThe unsupervised part of the learning proceeds as follows. First, we considerall possible splits of a molecule into two constituents using the following simpleapproach: each of the bonds in the molecule is removed in turn, and an algorithmis run to detect whether all atoms remain connected, or the result is a split intotwo. In the latter case, the viability of this split is tested by searching for analogysquares supporting that split. If more than one split is possible, the one with thehighest level of support is kept.The search for analogy squares is computationally very expensive. We havehowever developed a tailored concurrent GPU-powered algorithm to speed upthis computation. The algorithm takes advantage of a matrix representation ofa list of molecules with each row corresponding to a molecule, and each columncorresponding to a specific chemical element, e.g.:K Na O HKOH 1 0 1 1NaOH 0 1 1 1Individual constituents produced by a split can be represented in the same way.The operations of considering combinations of known constituents in order tocheck whether they form a known molecule (i.e. one from the list) can be repre-sented as an addition of matrices, which is easy to parallelise.Ultimately, given a list of N molecules, the result is a list of up to N pairs ofconstituents generated in this way. These splits can now be used to learn ILPsegmentation rules in same way as was done for word morphology [1].Another refinement is however possible before the ILP supervised learningstep is applied. One can use a simple Expectation Maximisation (EM) procedureto take into account the full list of segmented molecules in order to refine thelist of splits. The EM procedure alternates between the following two steps:1. A dictionary of all constituents appearing in the data is created, which alsocontains the relative frequency of each constituent, e.g. from Na+OH, K+OHwe generate: Na:0.25, K:0.25, OH:0.5.2. For each molecule, the likelihood of each possible split is estimated as theproduct of the relative frequencies of the two constituents, and the one withthe highest likelihood is kept.These two steps are repeated until convergence is reached.4 Experimental ResultsWe have four data sets of increasing complexity and size of the order of hundredsof examples extracted by an expert from the ChemSpider database.Fig. 1 is a sample of four molecules, where Col. 1 shows a chemical formula,Col. 2 uses the SMILES representation of the same compound, and Col.3–4 list4 The 29th ILP conference, 3–5 Sep 2019, Plovdiv, Bulgaria.pairs of constituents generated by the analogy squares procedure applied to alonger list of molecules containing these four. As typical for the SMILES repre-sentation, all hydrogen atoms have been omitted. Note the two alternative splitsTable 1. Output of analogy squares procedureFormula SMILES Const.1 Const.2C3NO CC(=O)NC C2 CNOC2NO CNCO C CNOC4O C=CCC=O C C3OC4O C=CCC=O C2 C2OC3O CCOC C C2Ofor the same molecule in rows 3–4. After the application of the EM procedure,only one split is left for each molecule, while the alternative split in Row 3 iseliminated.5 DiscussionThe first results show that the idea borrowed from word morphology produces re-sults that appear plausible on inspection by a domain expert. The parallelisationof the computationally heavy search for analogy squares makes the processing ofmuch larger data sets potentially viable, and the ways it scales up will be testedonce suitable data sets are prepared. We believe applying supervised learning tothese results shows great promise, and intend to apply ILP to the task as we didwith learning word constituents (and word segmentation rules) in the past.References1. Kazakov, D. and S. Manandhar. Unsupervised learning of word segmentationrules with genetic algorithms and inductive logic programming, Machine Learning,April-May 2001, vol. 43, pp. 121-162.2. Kazakov, D., Achievements and prospects of learning word morphology with in-ductive logic programming. In James Cussens and Saso Dzeroski, editors, LearningLanguage in Logic, pp. 89–109. Springer, 2000. Lecture Notes in Computer Science.Vol. 1925.3. Saussure, F. de: Cours de linguistique générale (1916)

Unsupervised Learning of Functional Groups for Computational Chemistry

https://eprints.whiterose.ac.uk/157651/1/ILP2019_Chemistry_abstract%20%281%29.pdf

Unsupervised Learning of Functional Groups for Computational Chemistry

Abstract

Similar works

Full text

Available Versions

White Rose Research Online